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ABSTRACT 

. The 5-year impacts of mandatory welf are-to-work programs on 
welfare recipients and their children were examined by using a rigorous 
research design called a social experiment to examine 11 welf are-to-work 
programs in 6 states (California, Georgia, Michigan, Ohio, Oklahoma, and 
Oregon) . Four employment-focused and seven education-focused programs were 
examined. Data were collected from administrative records, state and county 
welfare payment records, and surveys of mothers and children over the 5-year 
study. In the absence of welf are-to-work programs, approximately three- 
fourths of single-parent welfare recipients found jobs and more than half 
left the welfare roles. Although few of the 11 programs improved on that 
already-high rate of job finding, nearly all the programs helped single 

O 

ERIC 



Reproductions supplied by EDRS are the best that can be made 
from the original document. . 



parents work more hours during more quarters of the follow-up period and earn 
more than they would have in the absence of a program. The most effective 
program used an employment-focused approach that initially assigned some 
enrollees to very short-term education and training and others to job search. 
Impacts for children differed more by program site than by welfare-to-work 
approach. The following items are' appended: table and figure notes; 
supplementary tables; a survey response analysis; a comparison of impacts 
estimated from different data sources; and selected impacts for various child 
survey samples. (Contains 121 tables and 282 references.) (MN) 
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Executive Summary 



For the past 30 years, federal and state policymakers have been legislating various types 
of programs to increase employment among welfare recipients. How people can best move from 
welfare to work, however, has been the subject of long-standing debate. This report, summariz- 
ing the long-term effects of 1 1 mandatory welfare-to-work programs on welfare recipients and 
their children, represents a major advance in resolving this debate. The findings are the final ones 
from the National Evaluation of Welfare-to-Work Strategies (NEWWS), a multi-year study of 
alternative approaches to helping welfare recipients find jobs, advance in employment, and leave 
public assistance. 

“What works best, and for whom?” is the central question animating this report and the 
NEWWS Evaluation as a whole. In particular, the evaluation compares the effects of two alterna- 
tive pre-employment strategies, for different groups of welfare recipients: programs that empha- 
size short-term job search assistance and encourage people to find employment quickly (referred 
to as “Labor Force Attachment” [LFA] or, more broadly, “employment-focused” programs); and 
programs that emphasize longer-term skill-building activities, primarily basic education (referred 
to as “Human Capital Development” [HCD] or, more broadly, “education-focused” programs). 
The effects of each approach are estimated from a wealth of data pertaining to over 40,000 single 
parents (mostly mothers) and their children, and a five-year follow-up period (falling somewhere 
between 1991 and 1999, depending on the site), using an innovative and rigorous research design 
based on the random assignment of individuals to one or more program groups (with services) or 
to a control group (without services). 

I. Findings in Brief 

The research designs that were implemented in the NEWWS Evaluation permit many 
comparisons. The key ones examined the programs’ economic effects on adults and the “spill- 
over” effects on noneconomic outcomes and child well-being, as summarized below. 

Comparing All 11 Programs to What Would Have Happened in the Absence 

of the Programs 

• In the absence of any welfare-to-work program over a five-year follow-up pe- 
riod, approximately three-quarters of single-parent welfare recipients found 
jobs, and more than half left the welfare rolls. Few of the 1 1 studied programs 
improved on this already-high rate of job-finding, but nearly all programs 
helped single parents work during more quarters of the follow-up and earn 
more than they would have in the absence of a program. Moreover, all pro- 
grams decreased welfare receipt and expenditures over the five years. 

• Measured combined income, however, was largely not affected: The programs 
led to individuals’ replacing welfare and Food Stamp dollars with dollars from 
earnings and Earned Income Tax Credits (EITCs), but the programs did not 
increase income above the low levels of the control group. 
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• The programs achieved their economic gains with few spillover effects on 
such family measures as marriage, fertility, and household composition. Nota- 
bly, the adults’ gains in self-sufficiency (defined as increased employment and 
decreased welfare receipt) were achieved with few indications of harm or 
benefit to the well-being of their children. This was particularly true for moth- 
ers with young children, who in 1988 were newly mandated to participate in 
programs. Because the new mandate’s implications for children were of con- 
siderable concern at the time, these families were the subject of intense study 
in this evaluation. 

Comparing Labor Force Attachment (LFA) and Human Capital Development 

(HCD) Programs 

• By rigorously comparing LFA and HCD programs — versions of employ- 
ment-focused and education-focused programs designed to magnify the differ- 
ences between the two types of strategies and operated side by side in three 
evaluation sites — it was found that the HCD approach did not produce added 
economic benefits relative to the LFA approach. 

• Moreover, the LFA approach moved welfare recipients into jobs more quickly 
than did the HCD approach — a clear advantage when federally funded wel- 
fare months are time-limited. 

• Finally, the LFA approach was much cheaper to operate than the HCD ap- 
proach and, at the same time, did not affect sample members’ overall finan- 
cial well-being or their children’s well-being any differently than the HCD 
approach. 

• Surprisingly, these findings held true for program enrollees who lacked a high 
school diploma or a General Educational Development (GED) certificate as of 
study entry — the subgroup of welfare recipients who were expected to derive 
the greatest benefit from an initial investment in basic education — as well as 
for those who already possessed these education credentials. 

Comparing Employment-Focused and Education-Focused Programs 

• Dividing all 1 1 programs into two broad categories — employment-focused 
programs and education-focused programs — programs in the former category 
generally had larger effects on employment, earnings, and welfare receipt than 
those in the latter category. 

• Given the large number of programs examined and their variety of served 
populations, implementation features, and labor markets, these results provide 
more support for the advantages of employment-focused programs than for 
education-focused ones. 

These results should not be taken as an indictment of the benefits of education and train- 
ing in general in welfare-to-work programs. Nonexperimental work done as part of the NEWWS 
Evaluation has suggested that obtaining a GED and, especially, obtaining a GED and then receiv- 
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ing some type of vocational training, can result in employment and earnings gains for those who 
achieve these milestones.' However, in the context of mandatory welfare-to-work programs, few 
people make it this far, for many reasons, including: people leave welfare and therefore do not 
stay in welfare-to-work programs, and thus education or training classes, for very long; adults 
supporting families cannot afford an up-ffont deferment of employment and earnings that may or 
may not have a longer-run payoff; and only a small minority of welfare recipients report that, if 
given a choice, they prefer to go to school to study basic reading and math over going to school 
to learn a job skill or going to a program to get help looking for a job. ^ It should be noted as well 
that none of these programs made assignments to or emphasized college. 

The Features of the Most Effective Program 

• One program — the Portland (Oregon) one — by far outperformed the other 
10 programs in terms of employment and earnings gains as well as providing a 
return on every dollar the government invested in the program. 

• The Portland employment-focused program, unlike either the LFA or the HCD 
programs or the other education-focused programs, initially assigned some en- 
rollees to very short-term education or training and others (the majority) to job 
search. Also, in another departure from the other programs, job search partici- 
pants in Portland were counseled to wait for a good job, as opposed to taking 
the first job offered. While other aspects of the Portland program, such as its 
use of job developers and staffs experience operating job search programs, 
were also noteworthy, these distinctive features, along with other past re- 
search, suggest that a “mixed” approach — one that blends both employment 
search and education or training — might be the most effective. 

Findings for Children 

• Considering the six programs (three sites) in which children who were pre- 
school age at random assignment were studied in depth, impacts were found 
on a small number of measures of child well-being — predominantly in the 
area of the young children’s social skills and behavior. Overall, the young- 
child impacts differed more often by site than by welfare-to-work approach. 

• Program effects on child care — one important way in which children might 
be affected by welfare-to-work programs — diminished from the two-year fol- 
low-up point to the end of the five-year follow-up. As of this latter point, only 
the Portland program was still producing an increase in the use of child care. 



'Johannes M. Bos, Susan Scrivener, Jason Snipes, and Gayle Hamilton, Improving Basic Skills: The Effects of 
Adult Education in Welfare-to-Work Programs (Washington, DC: U.S. Department of Education, Office of the Un- 
der Secretary and Office of Vocational and Adult Education; and U.S. Department of Health and Human Services, 
Administration for Families and Office of the Assistant Secretary for Planning and Evaluation, 2001). 

^See Gayle Hamilton and Thomas Brock, The JOBS Evaluation: Early Lessons from Seven Sites (Washington, 
DC: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the 
Assistant Secretary for Planning and Evaluation; and U.S. Department of Education, Office of the Under Secretary 
and Office of Vocational and Adult Education, 1994). 



• In the seven programs (four sites) in which a limited number of measures were 
examined for children of all ages, few effects were evident. Some impacts, 
however, were found relating to young adolescents’ academic functioning (but 
in only two of the four sites for which data are available), and these impacts 
on adolescents were predominantly unfavorable. As was the case for young 
children, impacts on children of all ages did not differ by welfare-to-work 
program approach. 

Comparisons Shedding Light on Other Welfare-to-Work Program Design Issues 

• Of the two programs with low enforcement of the participation mandate, one 
had no impact on employment and earnings, and the other had only small ef- 
fects. It appears that a minimum level of enforcement by program staff is re- 
quired to produce at least moderate employment impacts, likely because this 
extra “push” is needed in order to engage in program activities those who 
normally would not participate on their own initiative. 

• Two of the three programs that used “integrated,” as opposed to “traditional,” 
case management worked well for those who entered the study without a high 
school diploma or GED. In integrated case management, one worker fulfills 
the responsibilities related to the payment of welfare and other benefits, nor- 
mally performed by income maintenance staff, as well as the responsibilities 
related to the provision of employment-related services, usually assigned to 
welfare-to-work program staff In traditional case management, each welfare 
recipient has two different case managers. Two programs that implemented 
different versions of well-funded and well-supported integrated case manage- 
ment produced relatively large impacts for nongraduates; the third program, 
which also used an integrated case management model but one that was ham- 
pered by tight funding, had limited impacts. 

The Limits of Pre-Employment Strategies 

Average income levels among control group members over the five-year follow-up period 
were low. Despite the successes of these programs, no program, not even Portland’s, met the 
long-range goal of making enrollees substantially better off financially. Most program group 
members continued to have low incomes from various combinations of earnings, the EITC, wel- 
fare, and Food Stamps. In fact, among individuals who lacked a high school diploma or GED as 
of study entry, some programs had the five-year result of making them financially worse off 
These findings suggest that the challenge of the future is to identify other types of programs or 
initiatives that can provide welfare recipients with better and more stable jobs, increase their in- 
come, and improve the well-being of their children. 

II. Background 

In 1988, the federal Family Support Act (FSA) established a system of mutual obligation 
within the Aid to Families with Dependent Children (AFDC) benefit entitlement structure: Gov- 
ernment was to provide education, employment, and support services to AFDC recipients, who in 
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turn were required to participate in the Job Opportunities and Basic Skills Training (JOBS) pro- 
gram created under that act. Through its mandates and incentives, the FSA encouraged state and 
local program administrators to serve welfare populations with whom they had had little if any 
contact in the past and to experiment with new types of services, messages, and mandates — of- 
ten in advance of solid evidence of how well these innovations would work. Most single parents 
with children ages 3 to 5 (or ages 1 to 5 at the state’s option) were required to, enroll in welfare- 
to-work programs. In addition, the FSA mandated that programs reserve at least 55 percent of 
funds for services to welfare recipients who were deemed to be at greatest risk of long-term wel- 
fare dependency. Furthermore, the FSA required enrollees to participate in employment- 
preparation activities for as long as they remained on welfare and eligible for services. Case 
managers were expected to monitor participation and to use a variety of informal and formal re- 
sponses (including reductions of welfare grants) when enrollees did not attend. In addition, peo- 
ple were supposed to be assigned to additional activities if they completed participation in 
emplojTnent-preparation activities without finding a job. 

The expansion of welfare-to-work programs and the requirement to work with more dis- 
advantaged populations intensified a long-standing debate among program administrators and 
policymakers concerning how best to help welfare recipients — especially those facing serious 
barriers to employment — move from welfare to work. Rigorous research in. the 1980s demon- 
strated that job search programs sped up the entry of welfare recipients into the labor market. Of- 
ten, however, the jobs were neither long-lasting nor high-paying, and they did not increase family 
income. Furthermore, the programs generally did not benefit the most disadvantaged welfare re- 
cipients. During the years before passage of the FSA, administrators of welfare-to-work pro- 
grams in several states and localities (most notably, California) began implementing programs 
that emphasized up-front investments in basic education and skill development as an alternative 
to job search. FSA regulations accelerated this trend by requiring states and localities to offer a 
variety of employment-preparation activities, including job search, basic education (classes in 
adult basic education, GED preparation, regular high school, and English as a Second Language), 
and vocational training and post-secondary education. 

Proponents of education- focused programs argued that this approach offered the best 
chance of helping people — especially those who lacked a high school diploma or faced other 
barriers to employment — to get better and more stable jobs, increase their family’s income, and 
reduce returns to the welfare rolls. Some further hypothesized that education-focused programs 
would benefit children more than job search programs, because parents who attended education 
or training classes would become more involved in their children’s schoolwork and would serve 
as role models for succeeding in school. 

There was little evidence at the time, however, that large-scale mandatory education pro- 
grams for welfare recipients would achieve these goals. It was expected that programs that empha- 
sized education and training would engage participants for months and perhaps years longer than 
programs that emphasized short-term job search assistance. As a result, education and training pro- 
grams would be more costly to run than job search programs. To be considered cost-effective from 
a budgetary standpoint, these programs would have to produce savings in welfare and other benefits 
well in excess of what the less expensive job search programs would attain. Moreover, participants 
in education and training programs were expected to experience an initial “opportunity” cost (com- 



pared with participants in employment-focused programs) in the form of forgone earnings during 
the period when they were in the classroom rather than the workplace. It was hoped that participants 
in education- focused programs would make up for such forgone earnings as well as their later start 
in accumulating work experience and on-the-job skills by attaining better initial jobs and by ad- 
vancing more quickly once employed than would otherwise be the case. However, it was not clear 
whether the people who were expected to attend education and training would actually remain in 
school long enough to attain credentials or enhance their job skills. This issue was especially crucial 
for people entering welfare programs with low levels of educational attainment or without the 
minimum literacy and math skills needed for employment. 

The FSA also afforded program administrators an opportunity to address the shortcom- 
ings of low-cost job search services. Many job search programs before the FSA required partici- 
pants to look for work but provided little instruction on how to find employment. In contrast, 
from the late 1980s onward, programs increasingly assigned enrollees to organized group job 
clubs, whereby participants received instruction on finding job leads, filling out resumes, and 
conducting job interviews. Many programs followed classroom instruction with one or more 
weeks of supervised job search, during which they provided job leads and the use of phone 
rooms to contact employers. Over time, program operators added new features to job club curric- 
ula, including career exploration, life skills and time management instruction, and self-esteem- 
building exercises. Some programs actively marketed their job placement services to area em- 
ployers and engaged in job development activities to increase the pool of available jobs. Admin- 
istrators also invested in new ways to communicate a pro-work message, although (as will be 
discussed below) the types of messages differed. Finally, the FSA’s ongoing participation re- 
quirement encouraged administrators to design follow-up activities (often short-term education 
and training) for job search participants who did not find employment. 

In general, implementing these enhancements made job search activities more costly to 
operate, but not as costly as education-focused programs. It was hoped that these changes would 
be cost-effective by helping more disadvantaged welfare recipients find work and by helping 
people find employment sooner than they would have otherwise. 

The most recent federal welfare reform effort, the 1996 Personal Responsibility and Work 
Opportunity Reconciliation Act (PRWORA), replaced AFDC with a flexible, state-directed block 
grant program. Temporary Assistance for Needy Families (TANF); set lifetime limits on most 
families’ receipt of federally funded TANF assistance; and created financial incentives for states 
to run mandatory, work-focused, welfare-to-work programs. If anything, TANF’s time limit, its 
focus on work, and its requirement that agencies work with the entire welfare caseload brought a 
new urgency to the question of which welfare-to-work approach was most effective. 

In addition, both the FSA and PRWORA strengthened the requirement that single parents 
prepare for employment in exchange for welfare benefits, and they extended this mandate to par- 
ents with young children. These developments increased the importance of learning how welfare- 
to-work programs affected families. To what extent would programs be able to involve mothers 
who had young children? Would parents with toddlers and preschool-age children be able to find 
stable, affordable, and high-quality child care while they participated in pre-employment activi- 
ties and while they were working? Would child care prove a financial burden and a barrier to 
employment after parents left government assistance? More generally, how would children be 
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affected if there were program-induced changes in their mothers’ educational attainment, hours 
of employment, or self-esteem; in their family income; or in the amount, type, or quality of child 
care they experienced? 

It was within this context that the NEWWS Evaluation was conceived and funded, in 1989, 
by the U.S. Department of Health and Human Services (HHS), with support from the U.S. Depart- 
ment of Education. The Manpower Demonstration Research Corporation (MDRC) conducted the 
evaluation. Child Trends, as a subcontractor, conducted the Child Outcomes Study, the part of the 
evaluation that examined effects on young children. 

III. Program Approaches and Implementation Features 

The programs in the NEWWS Evaluation implemented many of the features described 
above. As shown in Table 1, the 11 programs in the NEWWS Evaluation were operated in 
seven sites across the country. Employment-focused programs were operated in Atlanta, Geor- 
gia; Grand Rapids, Michigan; Riverside, California; and Portland, Oregon. Atlanta, Grand 
Rapids, and Riverside also operated education-focused programs, as did Columbus, Ohio (two 
programs); Detroit, Michigan; and Oklahoma City, Oklahoma. The studied programs were ini- 
tially administered under the FSA, which created the national JOBS program for recipients of 
cash assistance under AFDC. The programs continued (with some modification) under the 
ESA’s successor, PRWORA, which replaced AFDC with the TANF block grant program. Un- 
der both welfare reform acts, the programs’ primary goal was to move welfare recipients off 
government assistance and into paid work. 

The four employment-focused programs — the LFA programs in Atlanta, Grand Rapids, 
and Riverside as well as Portland’s program — assigned most enrollees to job club as their first 
activity, and they encouraged enrollees to find work as quickly as possible. Further, both Port- 
land’s and Riverside’s program employed full-time job developers to help place program enrol- 
lees in unsubsidized jobs. 

In contrast to the three LFA programs, however, Portland’s program offered GED prepa- 
ration classes to people who case managers thought had a good chance of attaining a GED cer- 
tificate relatively quickly. Furthermore, Portland case managers, more often than those in the 
LFA programs, encouraged enrollees to hold out for jobs that paid well above the minimum wage 
(about 25 percent higher) and that offered the best chance for long-lasting and stable employ- 
ment. Case managers in the LFA programs, especially Riverside’s, stressed the value of starting 
off with any job, even a low-paying one, and then advancing toward more stable and better- 
paying jobs in the future. 

The HCD programs in Atlanta, Grand Rapids, and Riverside; the Columbus Integrated 
and Traditional case management programs;^ and the programs in Detroit and Oklahoma City 



^For a full discussion of the results of the direct test of integrated versus traditional case management in the Co- 
lumbus site, see Susan Scrivener and Joharma Walter, Evaluating Two Approaches to Case Management: Implemen- 
tation, Participation Patterns, Costs, and Three-Year Impacts of the Columbus Welfare-to-Work Program (Wash- 
ington, DC: U.S. Department of Health and Human Services, Administration for Children and Families and Office of 

(continued) 
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National Evaluation of Welfare-to-Work Strategies 
Table 1 

NEWWS Programs, Categorized by Approach, First Activity, 
and Enforcement Level 



Employment-focused approach 


Education-focused approach 


Job search first 


Varied first activity 


Education or training first 


High enforcement 


High enforcement 


High enforcement 


Low enforcement 


Atlanta LFA 
Grand Rapids LFA 
Riverside LFA 


Portland 


Atlanta HCD 
Grand Rapids HCD 
Riverside HCD 
Columbus Integrated 
Columbus Traditional 


Detroit 

Oklahoma City 



NOTES: “LFA” denotes the site’s Labor Force Attachment program. 

“HCD” denotes the site’s Human Capital Development program. 



can each be characterized as “education-focused.” A large percentage of program enrollees in 
these programs were initially assigned to some type of skill-building activity. The types of ac- 
tivities to which enrollees were first assigned depended, in part, on the level of educational at- 
tainment that individuals had achieved prior to entering the program. Those who had not com- 
pleted high school or received a GED certificate but who were assessed by case managers as 
having high-school-level skills were assigned to GED preparation classes. Those with lower 
reading or math levels were assigned to adult basic skills classes. In addition, "non-English 
speakers could be assigned to English as a Second Language (ESL) programs. Finally, those 
who had completed high school or held a GED certificate could be assigned to vocational 
training or employment-oriented skills courses at local community colleges. All in all, how- 
ever, assignments to GED preparation or basic education courses predominated in these educa- 
tion-focused programs, and assignments to vocational training programs were less common, 
primarily as a result of welfare recipients’ low levels of educational achievement; enrollment 
in college played an even smaller role. 

Other key program features varied across the 1 1 studied programs as well. All four em- 
ployment-focused programs and five of the seven education-focused programs can be considered 
“high enforcement” programs: They worked with a cross-section of the welfare applicants and 
recipients who were required to participate; monitored participation closely; and, especially in 



the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education, Office of the Under Secre- 
tary and Office of Vocational and Adult Education, 2001). 



the two programs in Columbus and the two in Grand Rapids, frequently invoked sanctions (re- 
ductions in welfare grants) for nonparticipation. The remaining two education-focused programs, 
in Detroit and Oklahoma City, did not have these characteristics (because of either lack of funds 
or program philosophy) and can be considered “low enforcement” programs. 

The programs also differed in their child care policies and practices (within each site, 
however, child care assistance policies were identical for program and control group members). 
During the early to mid 1 990s, the Atlanta, Oklahoma City, Portland, and Detroit programs pro- 
vided the strongest staff support for arranging for child care, and the programs in Atlanta and 
Oklahoma City emphasized the use of licensed care; in contrast, case managers for both River- 
side programs encouraged enrollees to find low- or zero-cost, informal child care. 

The programs also differed in their case management strategies. Three programs — Co- 
lumbus Integrated, Portland, and Oklahoma City — implemented an integrated case management 
staffing arrangement. The other programs used a traditional case management structure. 



IV. Research Designs and Samples 

The NEWWS Evaluation used a rigorous design — called a social experiment — to esti- 
mate the effects of employment- and education-focused programs. Welfare recipients were ran- 
domly assigned to one of two or three research groups, depending on the site. 

As part of a largely unprecedented effort to determine which welfare-to-work program 
approach works better, three sites — Atlanta, Grand Rapids, and Riverside — simultaneously 
operated, expressly for the evaluation, two different programs: an LEA program and an HCD 
program. These programs were multidimensional but varied in terms of the key features that pro- 
gram operators and researchers thought most clearly differentiated employment- and education- 
focused programs. Each type of program communicated a different message to welfare recipients 
about the best route to employment, and each type differed from the other in the way program 
services were sequenced and emphasized. The programs were, however, mandatory to the same 
degree: Nonparticipants risked a reduction in their monthly welfare grant. In these three sites, 
welfare recipients were randomly assigned to an LEA program group, an HCD program group, or 
a control group. (Control group members were not subject to the participation mandate and re- 
ceived no services tlirough either type of program but, on their own, could seek out similar ser- 
vices within the community.) This random assignment research design produces the most reliable 
comparison between employment- and education-focused programs. It ensured that, within each 
site, there were no systematic differences between the background characteristics of people in the 
LEA, HCD, and control groups within each site when they entered the study. Thus, any subse- 
quent differences in outcomes between groups — comparing either the LEA or the HCD group to 
the control group, or the LEA and the HCD groups to each other — can be attributed with confi- 
dence to the effects of a particular type of program. These differences, referred to as the pro- 
grams’ impacts, are the primary focus of this report, and all differences reported are statistically 
significant unless otherwise noted. 

In the Columbus site, a three-group random assignment design was used as well. Here, 
the two program groups represented two case management models: “integrated” and “tradi- 
tional.” The remaining three sites in the evaluation — Detroit, Oklahoma City, and Portland — 
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used random assignment to test the effectiveness of established programs, as opposed to pro- 
grams designed to meet research protocols; individuals were randomly placed either in a group 
that entered the program or in a no-program control group. Note that control group members in 
all sites were eligible for child care assistance, similar to that offered to program group members, 
if they were participating in nonprogram activities in which they had enrolled on their own. 

In each site, individuals were randomly assigned to research groups over approximately a 
two-year period. Random assignment for the evaluation began in June 1991, in Riverside, and 
ended in December 1994, in Portland. Thus, the five-year results presented in this report cover 
the calendar period of June 1991 (the first sample member’s entry into the study) through De- 
cember 1999 (the last month of the five-year follow-up for the last sample member randomly as- 
signed, in Portland). 

The research designs that were set up in the seven NEWWS Evaluation sites permit many 
comparisons. The key ones examine the programs’ economic effects on adults and the spillover 
effects on noneconomic outcomes and children. The central comparisons may be expressed as 
follows: 

• Compared with what would normally happen in the absence of any type of 
welfare-to-work program, how effective are employment- and education- 
focused programs and, more narrowly, LEA and HCD programs? That is, 
comparing outcomes for the program groups with outcomes for the control 
groups, what are these programs’ net impacts? 

• Compared with one another, which approach is more effective? For example, 
comparing outcomes for the LFA and HCD groups directly (ignoring the con- 
trol groups), what are the differential impacts of the two approaches? 

• How effective are the programs for two key subgroups of welfare recipients 
for whom employment- and education-focused approaches might be ex- 
pected to work differently — namely, those who, as of study entry, had a 
high school diploma or GED (“graduates”) and those who did not (“non- 
graduates”)?"* 

It should be noted that while control group members were not exposed to the services and 
mandates of the sites’ programs for the first three years of follow-up, their status differed by site 
in the fourth and fifth years. For a few programs, net impacts for years 4 and 5 are understated 
somewhat because a small portion of the control group (a subset of those still on welfare) re- 
ceived program services toward the end of the five-year follow-up period. (In these sites, the 
“start of the clock” for welfare time limits necessitated allowing control group members access to 
welfare-to-work program services.) Importantly, however, this situation does not affect the re- 
sults of direct comparisons of the LFA and HCD program approaches (that is, the differential 



‘*The Riverside HCD program enrolled only individuals who did not have a high school diploma or GED, had 
low scores on baseline reading and math tests, or did not speak English. Comparisons throughout this summary be- 
tween the Riverside HCD and LFA programs or between the HCD and control group members include only such 
individuals, and they are referred to as “nongraduates.” 



impacts). The three-group random assignment designs in the three sites in which the LFA and 
HCD programs were operated side by side permit a direct comparison of these two approaches, 
that is, one that does not need to take into account the services received by, or the behavior of, 
control group members. 

This report includes data from administrative records (unemployment insurance [UI], 
state and county welfare payments, and Food Stamp data) and from surveys administered to 
mothers and children over the five years after individuals entered the study. 

V. Five-Year Effects on Use of Employment-Related Services and Costs 

• All programs increased participation in employment-related activities rela- 
tive to control group levels of self-initiated activity. Employment-focused 
programs produced large increases in participation in job search activities, 
and education-focused programs produced large increases in participation in 
basic education classes. 

Over the five-year follow-up period, a majority of control group members in each site (up 
to 75 percent) participated in some type of employment-related activity: job search, basic educa- 
tion, vocational training, or post-secondary education. Almost all of this activity was the result of 
control group members’ own initiative; despite the potential in several sites for controls to be 
subject to mandatory welfare-to-work programs at the end of the follow-up period, there is little 
evidence that much control group participation in such programs did, in fact, occur. Some of this 
self-initiated activity took place while control group members were receiving welfare; much of it 
took place after they left the welfare rolls. In most sites, the most common activities in which 
control group members enrolled themselves over the five-year period were vocational training 
and post-secondary education programs. 

All programs increased overall participation levels above those achieved by control group 
members. The employment-focused programs increased participation in job search by approxi- 
mately 30 percentage points relative to control group members. The education-focused programs 
increased job search participation as well, but to a much lesser degree. Most education- focused 
programs produced large increases in education and training relative to control group members. 
Among those who entered the study without a high school diploma or GED (“nongraduates”), 
increases were particularly large in basic education. Among those who entered the study with 
these credentials (“graduates”), few programs increased participation in vocational training. 
(While education- focused programs most commonly assigned graduates to vocational training 
activities, rarely did program group levels of participation in such activities exceed those of the 
control groups.) Increases in education and training participation in the employment-focused 
programs were much less common and, where they did occur, were smaller than those in the 
education-focused programs. 

The Portland program, with its employment focus but mix of initial program activity as- 
signments, produced five-year increases in both job search and education participation. While the 
program produced large increases in job search participation for both nongraduates and gradu- 
ates, it also resulted in a 10 percentage point increase (though not a statistically significant one) 
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in basic education participation among nongraduates and a large, 2 1 percentage point increase in 
post-secondary education participation among graduates. 

Overall, program group members’ length of stay was longer in education and training ac- 
tivities than in job search activities. For example, within the first two years of follow-up, the 
typical participant in an adult education program received the equivalent of about two-thirds of a 
year of instruction in a high school.’ Length of stay in any type of program activity was often cur- 
tailed because program group members started working for pay and/or left welfare — the goals, 
after all, of welfare-to-work programs. 

Education-focused programs generally increased the proportion of nongraduates who ob- 
tained a GED or high school diploma over the five-year follow-up period, whereas only the Port- 
land program among the employment- focused programs had such an effect (though this increase 
is not statistically significant). In addition, Portland had a notable increase in the proportion of 
nongraduates who obtained a high school diploma or GED as well as a second education or train- 
ing credential. Overall levels of high school diploma or GED receipt, however, were low: By the 
end of the five-year follow-up period, less than one-quarter of initial nongraduates in the educa- 
tion-focused program groups had obtained a high school diploma or GED. Among graduates, 
only the two programs in Atlanta produced five-year increases in the receipt of some type of edu- 
cation or training credential — generally, a trade license or certificate. 

• As expected, education-focused programs cost more than employment- 
focused programs over five years. Regardless of program approach, costs 
were higher for individuals who entered the study already possessing a high 
school diploma or GED (graduates) than for those who entered the study as 
nongraduates. 

The cost analysis considered all costs associated with providing employment services and 
associated support services to sample members (including case management costs). Costs paid by 
welfare departments and non-welfare agencies, and in-program as well as post-program or post- 
welfare costs, were included. 

Five-year net per-person costs (the gross cost per program group member minus the gross 
cost per control group member) averaged $3,037 for the employment-focused programs and 
$3,972 for the education-focused programs. (These are 1999 dollars.) The most reliable compari- 
son of the costs of these two types of program approaches, however, is one comparing LFA and 
HCD net program costs within each site. In Atlanta, Grand Rapids, and Riverside, HCD pro- 
grams were 40 percent to 90 percent more expensive than their counterpart LFA programs. 

NEWWS program costs were high compared with other programs that have been studied 
by MDRC. This is largely because of the greater use in all the NEWWS programs of high-cost 
education activities, such as post-secondary education and vocational training, relative to past 
welfare-to-work programs; as well as the enhancements made to job search activities by most 
NEWWS programs, relative to the above-described simple job search activities implemented in 
the 1980s. The average cost of the NEWWS programs was comparable to that of the two highest- 



’See Bos, Scrivener, Snipes, and Hamilton, 2001. 
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cost California Greater Avenues for Independence (GAIN) programs, which operated in Alameda 
and Los Angeles Counties in the late 1980s and early 1990s. 

VI. Five-Year Effects on Economic Outcomes for Adults 

A. Employment and Earnings 

• Most control group members worked at some point in the five-year follow-up 
period, without the assistance of a welfare-to-work program. 

At the high end, 88 percent of control group members (in Grand Rapids) were employed 
at some point during the five-year follow-up period; at the low end, 79 percent (in Oklahoma 
City) and 66 percent (in Riverside) worked during this same time frame. In addition to illustrat- 
ing the strong interest that welfare recipients have in going to work — regardless of any welfare- 
to-work program intervention — these figures suggest that there was little room left for programs 
to increase the proportion of program group members who “ever” worked. Employment levels 
for control group members grew steadily over the five-year follow-up period, although many 
control group members worked for less than one year and then experienced a spell of joblessness. 

• Nearly all 11 programs increased how much people worked and how much 
they earned, relative to control group levels, but the four employment- 
focused programs generally produced larger five-year gains in employment 
and earnings than did most of the seven education-focused programs. Port- 
land produced the largest, most consistent increases by far. 

Not surprisingly, given the high levels of employment for control group members, pro- 
grams generally had little effect on the percentage of sample members who “ever” worked. How- 
ever, in 9 of 1 1 programs, the program group worked during more calendar quarters on average 
than the control group; and in 9 of 1 1 programs, the program group averaged higher total earn- 
ings than their control group counterparts. 

Portland produced the largest, most consistent employment and earnings effects by far: 
Over five years, program group members worked 1.6 quarters more than control group members 
— a 21 percent increase in employment duration — and their average five-year earnings were 
about $5,000 greater than those of control group members. (See Figure 1, which depicts the im- 
pacts, or program-control differences, on earnings for all programs.) Portland’s program also 
produced the largest impacts on measures of stable employment and earnings growth among the 
11 programs. Portland’s success may have resulted from its imique combination of a focus on 
employment, the use of both job search and education, and an emphasis on finding good jobs. In 
addition, the program made extensive use of job development, and staff were experienced in op- 
erating welfare-to-work programs. Portland’s relatively strong economy also may have contrib- 
uted to the success of the program; however. Other programs in localities where the demand for 
labor was similarly high did not do as well. 

The employment-focused LFA programs in Atlanta, Grand Rapids, and Riverside also 
affected employment and earnings, but less so than the Portland program. Five-year earnings 
gains ranged from about $1,500 in Grand Rapids to about $2,500 in Atlanta and Riverside. The 
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National Evaluation of Welfare-to-Work Strategies 
Figure 1 

Program Impacts on Total Earnings in Years 1 to 5 
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programs also increased employment duration by an amoimt ranging from 0.7 quarter in Grand 
Rapids to 1.1 quarters in Riverside. 

The effects of the seven education-focused programs, as a group, were smaller than the 
effects of the employment-focused programs. Neither of the two programs with low enforcement 
of the participation mandate (Detroit and Oklahoma City) significantly affected employment. 
Among the other five education-focused programs, employment duration gains over five years 
ranged from 0.3 to 0.8 quarter, and earnings gains ranged from about $800 to about $2,000. 

Employment-focused programs produced effects almost immediately, whereas education- 
focused programs generally did not have effects until more than a year after random assignment. 
In the middle of the follow-up period, most of the programs increased employment and earnings, 
but effects diminished during the final two years and were statistically insignificant for most pro- 
grams by the end of year 5. (An example of this trend for the Grand Rapids LFA program can be 
seen by comparing the black bars and the white bars in Figure 2, and for the Grand Rapids HCD 
program by comparing the shaded bars and the white bars.) These results were especially disap- 
pointing for education-focused programs. As discussed above, education and training services 
were intended to help program group members eventually move into more stable and higher- 
paying jobs (compared with control group members and compared with those subject to em- 
ployment-focused programs) in order to make up for forgone earnings early in the follow-up pe- 
riod. However, most programs — education- or employment-focused — had little or no effect on 
measures of stable employment and earnings growth. 

A comparison of earnings impacts for nongraduates demonstrates more clearly the disap- 
pointing results for education-focused programs. For this subgroup, only two of the seven educa- 
tion-focused programs significantly raised five-year average earnings above control group levels, 
whereas three of the four employment-focused programs did so. The employment-focused pro- 
gram with the largest earnings impacts for nongraduates was Portland, however, which used a 
mix of initial activities that resulted in substantial use of education by nongraduates. Further- 
more, the education-focused Columbus Integrated program led to earnings impacts among non- 
graduates that, along with those of the Grand Rapids LFA program, were the next-largest among 
all programs. This suggests that education activities, in some instances, may contribute to earn- 
ings impacts. 

Employment-focused programs, compared with education-focused ones, also more con- 
sistently increased the earnings of high school graduates above control group levels. Once again, 
however, the largest earnings impact for high school graduates was found for the Portland pro- 
gram, which substantially increased the use of post-secondary education among graduates. 

• Directly comparing the LFA and HCD programs in the three sites in which 
these programs were run side by side (thus using the most rigorous method 
for assessing the relative effectiveness of employment- and education-focused 
programs), employment and earnings levels over five years were largely simi- 
lar for the two types of programs. Where there were differences between the 
two types of programs — for early follow-up years or for a particular sub- 
group or outcome measure — they were in favor of the LFA approach. 
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NOTES; Asterisks (♦) denote statistical significance for LFA-control or HCD-control differences: * =10 percent; ** = 5 percent; 
Daggers (|) denote statistical significance for LFA-HCD differences: t “10 percent; tt ^ 5 percent; ftt ~ 1 percent. 




Cumulatively, over the five-year follow-up period, few LFA-HCD differences in em- 
ployment or earnings (that is, differential impacts) are found when both graduates and nongradu- 
ates are included in the calculations (see the top panel of Figure 1).* Year by year, however, there 
were some differences. The first set of bars in Figure 2, showing only the Grand Rapids site, il- 
lustrates these. LFA average per-person earnings were higher than those of the HCD sample 
members in at least year 1 in Grand Rapids as well as in Atlanta and Riverside. The gap between 
the two types of programs narrowed, however, in year 2 of follow-up in Grand Rapids and in At- 
lanta and in year 3 of follow-up in Riverside. In addition, there was one LFA-HCD difference for 
the fiall sample on the measure of average quarters employed: In Grand Rapids, the LFA group 
worked more quarters than did the HCD group. Notably, the HCD programs, relative to the LFA 
ones, did not produce more earnings growth over the follow-up period or increase the likelihood 
of employment in “good” jobs. Finally, the yearly trends suggest that the story would not change 
if longer follow-up were available. 

• Again directly comparing the LFA and HCD programs in the three sites in 
which these programs were run side by side, employment and earnings im- 
pacts were greater in the LFA programs than in the HCD programs among 
nongraduates. Among graduates, the two approaches produced similar im- 
pacts. 

As shown in the lower panel of Figure 1, two LFA programs and one HCD program pro- 
duced five-year earnings increases, relative to control groups, for nongraduates; as shown in the 
middle panel of this same figure, a different set of two LFA programs and one HCD program 
produced five-year net earnings impacts for graduates. 

Contrary to expectations, earnings impacts generally were larger for nongraduates in LFA 
programs than in HCD programs. (Proponents of the HCD approach anticipated that education- 
focused programs might be particularly effective for those without high school diplomas or 
GEDs, inasmuch as their lack of skills or credentials might inhibit employers fi'om offering them 
jobs. But this did not turn out to be the case.) Among this subgroup, LFA-HCD differences in 
five-year earnings were $920 in Riverside, $1,095 in Atlanta, and $1,945 in Grand Rapids. While 
only the Grand Rapids difference is by itself statistically significant, the average difference across 
the three sites is statistically significant at the 5 percent significance level. Furthermore, in no 
year of the follow-up were nongraduates’ average earnings higher in HCD programs than LFA 
programs; rather, earnings were generally higher in the LFA programs, with statistically signifi- 
cant differences found in the early years of follow-up in every site. (See the last set of bars in 
Figure 2 for an illustration of this pattern in Grand Rapids.) Among graduates, earnings impacts 
were very similar in the two types of programs. 

B. Welfare Receipt and Payments 

• The majority of control group members in all sites were off the welfare rolls 
as of the end of the five-year follow-up period, without the assistance of a 
welfare-to-work program. 

®Note that, in all the figures, “daggers” indicate statistically significant differences between the LFA and HCD 
impacts. 



The average control group member remained on assistance for about two to three years 
during the five-year follow-up period. Levels of welfare receipt fell steadily over time for control 
group members, reflecting “normal” welfare exits. Welfare receipt reached particularly low lev- 
els in Columbus and Portland, where less than 20 percent of control group members were receiv- 
ing a welfare payment at the end of year 5. 

• All programs reduced months on welfare and Food Stamps as well as welfare 
expenditures over five years, relative to control levels, with most programs 
leading to relatively large welfare savings. Welfare reductions were not con- 
sistently larger in the employment-focused programs than in the education- 
focused ones. 

All programs had an effect on the number of months that people received welfare. On av- 
erage, employment- and education-focused program group members received AFDC or TANF 
assistance for two to six fewer months than their control group counterparts. 

All programs also reduced total welfare payments below control group levels, and most 
produced savings of 10 percent or more (a historically large effect). (See Figure 3.) For many 
programs, welfare savings were larger and more persistent than earnings gains: Few programs 
continued to affect employment and earnings in year 5, but most programs continued to generate 
welfare savings at the end of year 5. This finding implies that some program group members who 
exited welfare for employment early in the follow-up did not return to assistance after leaving 
employment, even though they may have been eligible to do so. It is possible that the national 
welfare climate in the aftermath of the federal welfare reform legislation of 1996 contributed to 
this pattern, and since more program than control group members left welfare in the early years 
of follow-up (before enactment of the legislation), the climate may have had more of an effect on 
program than control group members. 

Welfare savings were generally larger for programs that had greater effects on earnings, 
but they varied for other reasons as well. Total payments were reduced more in higher-grant sites 
such as Riverside, Portland, and Grand Rapids and were reduced less in lower-grant sites such as 
Atlanta. In addition, welfare benefits were reduced more in sites that strictly enforced program 
participation mandates, such as Columbus and Grand Rapids, but benefits were reduced rela- 
tively little in sites that did not enforce mandates, such as Detroit. 

The programs had similar welfare impacts for high school graduates and nongraduates. 
Most programs produced welfare savings for both groups, and there is little evidence that the ef- 
fects were larger for one group than the other: In five programs, welfare savings were larger for 
graduates, but in five other programs, welfare savings were larger for nongraduates. 

Over five years, program group members in all programs spent less time on Food Stamps 
and on average received smaller Food Stamp payments than control group members. Food Stamp 
impacts were generally smaller than welfare payment impacts, however, because some program 
group members appropriately continued to receive Food Stamps after they left welfare. 

• In the three LFA-HCD sites, LFA sample members left welfare at a slightly 
faster pace than HCD sample members in the first year of follow-up, but the 
gap narrowed in subsequent years. Only in one site did the LFA and HCD 
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Figure 3 

Program Impacts on Total Welfare Payments in Years 1 to 5 
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programs differ with respect to the number of months on welfare or welfare 
expenditures over five years. In this site, welfare months and expenditures 
were lower in the LFA program than the HCD program. 

Cumulatively, over the five-year follow-up period, a statistically significant LFA-HCD 
difference (differential impact) in welfare expenditures was found in only one site (Grand Rap- 
ids), where the LFA program produced savings of $785 more than the HCD program (see Figure 
3) and where the average number of months in which welfare or Food Stamps were received also 
was lower in the LFA program. In Grand Rapids, this pattern held for graduates as well as non- 
graduates. In Atlanta, welfare expenditures and months on welfare were lower for the LFA pro- 
grams than for the HCD programs, but these differences are statistically significant only among 
the Atlanta nongraduates. In Riverside, the LFA and HCD programs led to similar reductions in 
welfare receipt and expenditures for nongraduates. The fact that in all three sites LFA sample 
members left welfare more quickly than HCD sample members represents a clear advantage in an 
environment where federally funded welfare months are time-limited. 

C. Combined Income 

• The combined income from earnings, welfare and Food Stamp payments, 
and Earned Income Tax Credits for control group members was low. On the 
positive side, over the five years, program group members received a larger 
portion of such combined income from earnings, compared with the control 
group. The programs, however, were largely unable to increase total com- 
bined income. Income impacts varied more by site than by program ap- 
proach, but, among nongraduates in the three LFA-HCD sites, those in the 
LFA program groups had higher combined income than those in the HCD 
program groups. 

Both employment- and education-focused programs helped sample members become 
more self-sufficient relative to the control group by increasing employment and earnings and re- 
ducing public assistance. As a result of these changes, program group members received a higher 
percentage of their income from earnings, compared with the control group. This impact aver- 
aged about 4 percentage points across all programs but was somewhat larger for employment- 
focused programs, especially the Portland one, than for education-focused programs. 

Program group members, however, received about the same amount of income as their 
counterparts in the control group. Two programs — one employment-focused and the other edu- 
cation-focused — led to especially large decreases in welfare and Food Stamp payments and to 
decreases in combined income. In general, program effects on combined income were less posi- 
tive (larger decreases or smaller increases) for nongraduates than for graduates. In addition, 
among nongraduates, LFA programs did better than HCD programs. Among nongraduates in the 
LFA-HCD sites, a simple average of the impacts across the three sites indicates that the three 
LFA programs as a group resulted in almost $1,000 more in combined income over five years 
than the HCD programs, a statistically significant difference. 

Including estimates of sample members’ Earned Income Tax Credits and payroll taxes did 
not change the above results. (These findings do not account for program effects on other possi- 



ble sources of income, such as child support payments or unemployment insurance benefits or 
income from spouses, partners, or other household members. The available data, however, sug- 
gest that the inclusion of these other income sources would have changed the impact estimates 
very little, if at all.) 

D. The “Most Disadvantaged” Subgroup 

• Neither employment-focused nor education-focused programs consistently 
had the largest earnings impacts for sample members who could be consid- 
ered the “most disadvantaged.” 

As discussed above, FSA programs were required to target welfare-to-work resources on 
individuals at greatest risk of long-term welfare dependency. The NEWWS Evaluation was de- 
signed to determine which types of employment-preparation services provide the greatest benefit 
to at-risk populations. 

In several respects, both employment- and education-focused programs were successful. 
Most programs raised earnings above control group levels for sample members with serious barri- 
ers to employment, such as no recent work history and a lengthy history of prior welfare receipt. 
Similarly, in most programs, the most disadvantaged group members — welfare recipients who did 
not have a high school diploma or GED, who had a history of welfare receipt, and who had not 
worked recently — earned more on average than their counterparts in the control group. Differences 
for several of these programs are small, however, and are not statistically significant. 

In Atlanta and Grand Rapids, the LEA programs, compared with the HCD ones, led to 
considerably higher earnings impacts for the most disadvantaged sample members — especially 
in Atlanta, where the HCD program had no effect on this subgroup. However, earnings increases 
for this subgroup (relative to the control group) were similar for the two programs in Riverside. 

Although the more disadvantaged groups had higher earnings as a result of most 
NEWWS programs, they still earned very little. In addition, most programs reduced welfare and 
Food Stamps by a larger margin than they increased earnings. As a result, programs did not raise 
the combined income of the most disadvantaged recipients above control group levels. 

VII. Benefit-Cost Analysis 

The benefit-cost analysis extends the findings on program impacts and costs presented 
above. It considers program effects on additional outcomes, such as fringe benefits from em- 
ployment, income and sales taxes, Medicaid expenditures, and the administrative costs of trans- 
fer programs. These additional outcomes were estimated or imputed from administrative records 
and published data. The analysis also considers program effects from the standpoint of sample 
members (referred to as the welfare sample perspective) and of government (referred to as the 
government budget perspective). 

The benefit-cost analysis from the welfare sample perspective considers (in a more com- 
prehensive way than presented above) whether programs increased program group members’ in- 
come — from any source — relative to the control group. (While society generally favors earn- 
ings over welfare payments, the benefit-cost analysis from the welfare sample perspective does 
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not favor either income source.) The welfare sample derives a net gain from a welfare-to-work 
program if the program increases earnings (plus fringe benefits) by an amount that exceeds the 
sum of the welfare, Food Stamp, and Medicaid benefits lost and the increase in taxes paid (net of 
Earned Income Tax Credits). 

From the government budget perspective, in contrast, increases in tax revenues and re- 
ductions in benefits are a net gain. The government budget perspective also counts as benefits 
any savings in administrative costs from reductions in receipt of transfer payments. These gains 
are compared with the program-control group difference in cost, that is, with the net cost of pro- 
viding employment-related services. 

Programs may lead to net gains from both the welfare sample and the government budget 
perspectives; they may also lead to net losses from both perspectives. Other times, programs may 
benefit either the welfare sample or the government budget. 

• From the benefit-cost perspective of the welfare sample, most of the pro- 
grams resulted in financial losses. From the perspective of government bud- 
gets, the majority of the programs saved the government about as much as 
they cost 

From the benefit-cost perspective of the welfare sample, most of the programs produced 
net financial losses. Moreover, gains were close to zero in the few programs that resulted in 
them. (See Table 2.) In contrast, from the benefit-cost perspective of government budgets, the 
majority of programs broke even (that is, saved the government only slightly more or slightly less 
than they cost), a few produced clear savings, and one produced a clear cost. Government budget 
savings in several programs were larger for nongraduates than for graduates. 

• Directly comparing the benefit-cost results for LFA and HCD programs 
shows that full-sample results were similar — from the perspective of the 
welfare sample — for both programs within each of the three sites. Non- 
graduates, however, uniformly experienced losses, which in each site were 
greater in the HCD programs than in the LFA programs. From the govern- 
ment budget perspective, returns to investments in each site were greater in 
LFA than HCD programs. 

From the perspective of welfare sample members, neither LFA nor HCD programs con- 
sistently yielded gains or losses. (See Table 2.) Rather, in each site, both programs produced ei- 
ther a net loss for all enrollees (in Grand Rapids and Riverside) or a gain close to zero (in At- 
lanta). Nongraduates, however, experienced losses over the five years in both types of programs, 
but the losses were consistently greater in the HCD than the LFA programs. For graduates, re- 
sults were mixed. 

From the standpoint of government budgets, neither LFA nor HCD programs consistently 
produced budget savings or losses. In every site, however, for all sample members as well as for 
nongraduates and graduates, five-year government budget savings were greater, or losses were 
smaller, for the LFA programs. (See Table 2.) The differences between the gains or losses for the 
LFA and HCD programs within each site were quite large, particularly in Atlanta and Grand 
Rapids. 
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diploma or GED 

Net saviiiRS (net cost) ($) (1,999) (5,700) 5,564 1,096 1,278 606 2,200 811 437 2,787 



• Compared with the returns to government budgets per net dollar invested in 
previously studied welfare-to-work programs, the NEWWS programs’ gov- 
ernment budget returns are similar, if not higher. 

To facilitate comparisons with the benefit-cost results of previously studied welfare-to- 
work programs, an additional measure of the cost-effectiveness of the NEWWS programs from 
the government budget perspective was used. This measure, called the return to budget per net 
dollar invested, is calculated by dividing the gains (taxes and savings in transfer payments and 
associated administrative costs) by the total net costs of services. Using this metric, government 
budgets come out ahead if programs produce more than a dollar’s worth of additional revenues 
and savings for each dollar spent on employment-related services for program group members 
(compared with control group members). 

Using this measure, the Portland program and the Grand Rapids LFA program both pro- 
duced over $2.00 in increased revenue and savings for every additional dollar spent on program 
group members. The Riverside LFA program also produced a considerable return, $1.47 per dol- 
lar invested. The Grand Rapids HCD and the Columbus Integrated programs essentially caused 
the government to break even ($0.92 to $1.06). The Atlanta, Detroit, and Columbus Traditional 
programs were not as successful, returning considerably less than one dollar for each dollar in- 
vested, ranging from $0.41 in the Atlanta HCD program to just over $0.80 in the other programs. 
The average return across all the programs was $1.29. 

On average, these results are more positive than those found in benefit-cost analyses of 
prior, recent programs. The California GAIN programs, for example, had returns to government 
budgets that ranged from a low of $0.17 per dollar invested (Tulare County) to a high of $2.84 
(Riverside County). The average across all counties in the GAIN evaluation was $0.76. The re- 
turn on investments in the two NEWWS programs that were most successful from a government 
budget perspective — Portland ($2.83) and Grand Rapids LFA ($2.46) — compare favorably 
with previously studied programs that had high government budget returns: the Riverside GAIN 
program and the mid-1980s San Diego Saturation Work Initiative Model (SWIM) program, 
which returned $2.84 and $2.34, respectively. 

VIII. Effects on Family Circumstances and Children’s Well-Being 

No aspects of the welfare-to-work programs studied as part of the NEWWS Evaluation 
were designed to directly change family circumstances — for example, to specifically affect mar- 
riage or fertility rates or to improve child well-being in specific ways. Theoretically, however, the 
programs could indirectly affect family circumstances or children through their impacts on such 
adult outcomes as educational attainment, employment, earnings, welfare status, and income. 
Data on five-year family and child outcomes are available for seven programs in four sites (At- 
lanta, Grand Rapids, Riverside, and Portland). The evaluation examined the net impacts on fam- 
ily circumstances and child well-being, that is, increases or decreases relative to the situations 
among control group members. 

• As of the end of the five-year follow-up period, no programs increased or de- 
creased adults’ or dependent children’s health care coverage. Only two pro- 
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grams — the ones in Riverside — increased the use of Transitional Medicaid 
at any point over five years. 

From 70 percent to 80 percent of sample members had health care coverage at the end of 
follow-up year 5, and most of them had coverage through public sources, such as Medicaid or 
other public programs, rather than through private sources, such as employers’ health plans. All 
of those who did not have coverage had left welfare. (Many received Transitional Medicaid but 
were not able to find alternative coverage when their transitional benefits expired.) None of the 
programs had an impact on health care coverage for adults or children, although most programs, 
because they increased employment, led to a shift fi"om public to private coverage. The two pro- 
grams in Riverside increased the use of Transitional Medicaid during the follow-up period, but 
neither program led to a gain in coverage as of the end of five years. 

• Over the five years, the programs had no effect on marriage rates and few ef- 
fects on household composition and living arrangements. The programs did 
lead to program-control group differences on measures of one aspect of the 
quality of relationships. In particular, program group members, compared 
with control group members, were less likely to report experiences with 
physical abuse during the last year of follow-up. There were no impacts on 
other measures of nonphysical abuse or job-related harassment 

The Grand Rapids LFA and HCD programs led to the largest effects on living arrange- 
ments. This site’s two programs increased the likelihood that people would move (and move 
more than once), primarily to attain better housing. In addition, the Grand Rapids HCD program 
increased home ownership. 

Information about abuse by intimate partners or others and about barriers to work put up by 
intimate partners or others is available for a subset of sample members — those who entered the 
study with a preschool-age child — in three sites (Atlanta, Grand Rapids, and Riverside), covering 
six programs. Similar percentages of program and control group members in these sites reported 
experiencing harassment, abuse (physical or otherwise), or other types of deterrence fi"om working 
at some time in their lives. More positive results, however, were found at the end of the follow-up 
period. During follow-up year 5, reported rates of any abuse by intimate partners among control 
group members ranged fi"om 19 percent to 22 percent. All six programs decreased reports of physi- 
cal abuse (such as hitting) by intimate partners, by 3 to 6 percentage points, although the decreases 
are statistically significant only in the Atlanta LFA, Grand Rapids HCD, and Riverside LFA pro- 
grams. There is some evidence that these reductions in reports of abuse were fostered by increases 
in employment (which may have increased individuals’ self-esteem or self-efficacy, ameliorated 
family stress, or simply reduced the amount of time that individuals spent with partners) and by 
program caseworker attention to support services. Notably, this evaluation did not try to identify 
women who might be in imminent danger related to abuse. For some women, work may lead to 
greater safety. For others, however, especially those in imminent danger of abuse, employment at a 
time of risk may not have such positive results, and such a risk may make it difficult for them to 
work or comply with welfare-to-work program requirements. 

• Program effects on the use of child care diminished over time. 
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As discussed above, both employment- and education-focused programs increased em- 
ployment levels and earnings during the early years of follow-up, but impacts grew smaller 
thereafter. The longer-term effects on use of child care for employment reflect these trends. 
During the first two years after random assignment, all four employment-focused programs 
plus three education-focused programs produced moderate to large increases in child care use 
while employed. However, at the five-year mark, only Portland’s program increased (by 7 per- 
centage points) child care use during sample members’ most recent job. (It should also be re- 
membered that most children who were studied intensively early in the follow-up were attend- 
ing school by the end of year 5.) 

A higher percentage of program group members received transitional child care benefits 
after random assignment, although the increase is statistically significant only for the two pro- 
grams in Atlanta and the Riverside LFA program. This increase occurred because a higher per- 
centage of program group members left welfare for employment (and became eligible for transi- 
tional benefits) and because program group members who became eligible for benefits more of- 
ten received them than their counterparts in the control group. 

The analysis also considered whether programs led to greater child care use for any pur- 
pose at the end of follow-up or otherwise altered how children spent their time. This issue was 
examined for a subset of sample members — those who entered the study with a preschool-age 
child — in six programs. Of these six, only one (the Riverside LFA program) had an effect on 
recent child care use — a decrease of 5 percentage points in the use of formal care. In addition, 
both programs in Riverside reduced the time that the 8- to 10-year-old children spent with their 
mothers, and these programs increased the time that these children spent with another adult. 

• The programs led to impacts on a small number of measures of child well- 
being among children who were preschool age at random assignment (for 
whom in-depth data are available). These impacts varied in size and direc- 
tion, and they varied by site more often than by welfare-to-work approach. 

The three-site Child Outcomes Study that was nested within the larger NEWWS Evalua- 
tion included nearly 50 measures of children’s academic functioning, health and safety, and so- 
cial skills and behavior for children who were preschool age at study entry in six programs, 
providing about 300 program-control group comparisons. About 15 percent of these tests yielded 
statistically significant differences — a relatively small percentage, but more than would be ex- 
pected by chance. 

Most commonly, each of the six programs affected young children’s social skills and be- 
havior, but in different ways. The two Atlanta programs led to favorable impacts: both a higher 
percentage of positive behaviors, such as being sensitive to others or making friends, and a lower 
percentage of negative behaviors, such as fighting or arguing with others. In contrast, the two 
Grand Rapids programs and the Riverside LFA program led to unfavorable impacts: decreased 
positive behaviors and/or increased problem behaviors. 

Impacts on young children’s academic achievement or academic performance were few. 
However, results for measures relating to behavioral adjustment to school (these include discipli- 
nary problems and the degi'ee of engagement in school) were consistent with the impacts on so- 
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cial skills and behavior. Both the Atlanta LFA and HCD programs led to favorable impacts (al- 
though none were statistically significant in the LFA program), whereas the Grand Rapids LFA 
and HCD programs led to unfavorable effects. 

Some negative results were also found for young children in Atlanta. Children in At- 
lanta’s program groups either missed or were late for school more often than their counterparts in 
the control group. Children of Riverside HCD program group members also averaged more days 
absent from school than children of control group members. There were few impacts on meas- 
ures of children’s health and safety; however, the effects were all unfavorable and occurred 
mostly in these same three programs. 

e Considering children of all ages, the programs led to few effects on children, 
although some impacts were found relating to young adolescents’ academic 
functioning. When found (in two of the four sites for which data are avail- 
able), the impacts on adolescents were predominantly unfavorable. 

The evaluation examined program effects on a limited number of measures of academic 
functioning and of health and safety for children in different age groups in four sites (encompass- 
ing seven programs). Few effects were found, and these did not vary consistently by program ap- 
proach or site. Some programs (in Grand Rapids and Riverside) led to some unfavorable impacts 
for young adolescents, but other programs (in Atlanta and Portland) led to few favorable or unfa- 
vorable effects for this same age group. In Riverside, for example, about 4 percent of adolescent 
children of control group members had ever repeated a grade in school, and both programs in this 
site increased this rate by 3 to 4 percentage points. In Grand Rapids, about 8 percent of control 
group adolescents had ever repeated a grade, and this rate increased by 4 percentage points as a 
result of both programs in this site. In Riverside, there were unfavorable effects on a few other 
measures as well, particularly in the site’s HCD program. For example, the Riverside HCD pro- 
gram increased the likelihood that an adolescent would drop out of school, and it increased the 
percentage of adolescents who had a physical, emotional, or mental condition that impeded their 
mother’s ability to go to work or school. In addition, both Riverside LFA and HCD programs — 
among families in which the parents lacked a high school diploma or GED — produced increases 
in the proportion of adolescents who had a baby as a teen. On the positive side, however, the At- 
lanta LFA program decreased the proportion of adolescents who had ever been suspended or ex- 
pelled from school; the Portland program had no effects on adolescents; and on many other 
measures, either none or only one of the programs had effects on adolescents. In general, it is 
possible that adolescents’ academic functioning may have been especially vulnerable to the in- 
creased employment, decreased income, and/or changes in household composition that occurred 
among their mothers in several of the programs. 



Chapter 1 

Introduction 



For the past 30 years, federal and state policyrhakers have been looking for new and better 
ways to increase the employment of welfare recipients. Beginning in the late 1960s, in response to 
dissatisfaction with the Aid to Families with Dependent Children (AFDC) program, one of the na- 
tion’s principal safety nets for poor families. Congress began to reshape it, creating a program to 
encourage welfare recipients to find jobs. In 1988 the Family Support Act (FSA) established a sys- 
tem of mutual obligation within the AFDC benefit entitlement structure: Government was to pro- 
vide education, employment, and support services to AFDC recipients, who were, in turn, required 
to participate in the Job Opportunities and Basic Skills Training (JOBS) program. The most recent 
federal reform effort, the 1996 Personal Responsibility and Work Opportunity Reconciliation Act 
(PRWORA), replaced AFDC with a flexible, state-directed block grant program. Temporary Assis- 
tance for Needy Families (TANF); limited most families to five years of lifetime federal TANF as- 
sistance (with some states setting shorter welfare time limits); and created financial incentives for 
states to run mandatory, work-focused welfare-to-work programs. PRWORA gives states more 
flexibility than they had before in designing their programs (which has encouraged, for example, 
some states to implement generous financial work incentives or tough financial sanctions for non- 
cooperation) and more responsibility for moving the nation’s poor into the labor market. 

Mandatory, work- focused welfare-to-work programs are thus not a new idea. The challenge 
for welfare policymakers and state and local program administrators, however, is to determine how 
to design and implement such programs so as to best achieve the goal of fostering adults’ long-term 
economic self-sufficiency without unintentionally jeopardizing the well-being of their children. 
This report provides guidance on the topic by analyzing the long-term effectiveness of 1 1 programs 
in seven sites that aimed to move substantial numbers of people fi'om welfare to work. These pro- 
grams were begun under FSA but operated under TANF in the later years of the evaluation. (The 
results in this report pertain to the time period between 1991 and 1999.) Overall, the programs 
shared TANF’s primary goal of moving people fi'om welfare to work. Further, they reflect a range 
of approaches, implementation features, and environments: Some were strongly employment- 
focused while others emphasized basic education; they varied in how broadly the program partici- 
pation mandate was applied to the welfare caseload and how strictly it was enforced, in the amount 
of child care support provided for program participation or employment, and in methods of case 
management; and the programs served different welfare populations and operated in a variety of 
labor markets. Finally, all 1 1 programs were studied using a strong random assignment design, re- 
sulting in reliable information about their relative effectiveness. 

This report analyzes the comparative long-term effectiveness of two very different types 
of programs: employment-focused programs and education-focused programs. Results fi'om the 
1 1 programs studied provide evidence on the relative effectiveness of these two approaches in 
two ways: First, an innovative research design implemented in three sites is used to directly com- 
pare versions of these two approaches within each site, resulting in unusually reliable informa- 
tion about their relative effectiveness. Second, cross-site comparisons of all 1 1 programs studied 
are used to determine if more examples of the two approaches, operating in different labor mar- 
kets and for different populations, uphold or refine the findings fi'om the direct comparisons in 
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three sites. In addition, the cross-site comparisons are used to ascertain if any versions of these 
approaches emerge as particularly successful. Results are presented for two key subgroups of 
welfare recipients for whom different program approaches might be expected to work differently: 
those with a high school diploma or GED and those without these credentials. Covering a five- 
year follow-up period and using a wealth of data pertaining to single parents (mostly mothers) as 
well as their children, the report thus addresses the following critical question: What works best 
and for whom? 

The five-year results presented in this report were produced as part of the National 
Evaluation of Welfare-to-Work Strategies (NEWWS). This evaluation is being conducted by 
the Manpower Demonstration Research Corporation (MDRC) under contract to the U.S. De- 
partment of Health and Human Services (HHS), with support fi'om the U.S. Department of 
Education. Child Trends, as a subcontractor, is conducting the Child Outcomes Study (COS), 
the part of the evaluation that examines effects on young children. The NEWWS Evaluation 
includes programs in seven sites across the country: Atlanta, Georgia (Fulton County); Grand 
Rapids, Michigan (Kent County); Riverside, California (Riverside County); Columbus, Ohio 
(Franklin County); Detroit, Michigan (Wayne County); Oklahoma City, Oklahoma (Oklahoma, 
Cleveland, and Pottawatomie counties); and Portland, Oregon (Multnomah and Washington 
counties).' In these seven locations, more than 55,000 people were randomly assigned to re- 
search groups as part of the study. 

This chapter begins by presenting a fi-amework for understanding the five-year results and 
then describes the research questions and design of the overall NEWWS Evaluation. Next, the 
environments in which the 1 1 programs studied were operated are discussed, along with the most 
critical program features. This is followed by an examination of the welfare-to-work program 
treatments experienced by individuals in the various evaluation research groups over time, given 
the long follow-up examined in the report. Finally, two-year NEWWS Evaluation results are re- 
capped, and the contents of Chapters 2-13 are briefly described. 

I. A Framework for Understanding Program Results 

The FSA gave program administrators a great deal of flexibility in designing the 1 1 
programs studied in the National Evaluation of Welfare-to-Work Strategies. (Box 1.1 provides 
a comparison of the key features of FSA and TANF.) That flexibility, combined with local 
economic, political, and funding environments, resulted in 1 1 programs that vary on several 
dimensions. This report focuses primarily on one of those dimensions: the self-sufficiency ap- 
proach used. 



'The programs and individuals studied in this evaluation are drawn from the entire county (or counties) men- 
tioned in parentheses after the city name; for ease of reference, in this report the sites will be referred to by the name 
of their corresponding urban area. 
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Box 1.1 



Key Features of the 

Family Support Act (FSA)/Aid to Families with Dependent Children (AFDC) 
and the Personal Responsibility and Work Opportunity Reconciliation Act 
(PRWORA)/Temporary Assistance for Needy Families (TANF) 



FSA/AFDC (1988-1996) 

• Entitlement to welfare, no time limit 

• Required participation of single parents with youngest child aged 3 or over (state op- 
tion: aged 1 or over) in welfare-to-work program activities for an average of 20 hours 
per week for most welfare recipients; other exemptions included being under age 16 
or over age 60, having an incapacitating illness, caring for an ill or incapacitated 
household member, or living in a remote area where services were not available 

• Expanded previous mix of job search-focused pre-employment services: many pro- 
grams emphasized skill-building activities 

• Limited per-person funding, given the size of the welfare caseloads 

• Targeted funding to people at risk of long-term welfare receipt 

PRWORA/TANF (1996-2002) 

• Time-limited federal support for welfare; block grants to states 

• Requires participation of single parents with youngest child aged 1 or over (state op- 
tion: even younger) in welfare-to-work program activities for an average of 30 hours 
per week, including at least 20 hours in actual work or job search; exemption criteria 
limited and at state discretion 

• Strongly encourages work-focused programs; limited opportunity for skill-building 
activities 

• Added funding and even more flexibility: many programs add post-employment ser- 
vices and stronger financial incentives to work 

• No targeting among recipients 
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While the overarching goal of programs in the past 30 years — to foster the self- 
sufficiency of welfare recipients through increased employment and decreased welfare receipt — 
has not changed, there has been disagreement on how best to move individuals fi'om welfare to 
work. One strategy emphasizes quick employment, reflecting the belief that individuals can best 
build their employability and improve their skills, eventually achieving self-sufficiency, through 
actual work, even if their initial jobs are minimum wage and without fiinge benefits. The other 
strategy emphasizes initial investments in short-term education and, in some cases, training, re- 
flecting the view that these investments will eventually enable individuals to obtain higher-wage, 
longer-lasting jobs with health insurance coverage. Most programs have blended the two strate- 
gies and emphasized elements of both. Past research has shown that a program’s location on the 
continuum between these two strategies and the mix of services it provides to enrollees can have 
an effect on the patterns and magnitude of program impacts measured in the short and long term.^ 

The programs in this report have been categorized by their approach: either employment- 
focused or education-focused. Three sites in the evaluation (Atlanta, Grand Rapids, and River- 
side) simultaneously implemented a Labor Force Attachment (LFA) program and a Human Capi- 
tal Development (HCD) program, versions of employment-focused and education- focused pro- 
grams that magnified the differences between the two types of approaches. The six programs in 
these three sites provide the best test of the relative effectiveness of the two approaches.^ Another 
site, Columbus, was also asked to implement two different programs in a head-to-head test. One 
program used an “integrated case management” staffing structure, in which one worker assumes 
responsibility for both eligibility and employment and training for her clients. The other program 
used a “traditional case management” staffing structure, in which separate workers handle the 
eligibility and employment and training duties. These programs, called the Columbus Integrated 
and Traditional programs, both used an education-focused approach."* Program administrators in 
the other three sites chose which self-sufficiency approach to implement based on their own 
goals. Of the 1 1 programs studied, four programs (Atlanta LFA, Grand Rapids LFA, Riverside 
LFA, and Portland) were employment-focused; the remaining seven were education-focused. 

In the three LFA versions of the employment-focused program, almost all enrollees were 
first assigned to job search. In Portland, the other employment-focused program, many, but not 
all, individuals were assigned to job search as a first activity. Some individuals, usually those 
who were determined to have more barriers to work than other members of the caseload, were 
first assigned to education or training activities. In the three HCD education-focused programs. 



^See Friedlander and Burtless, 1995; Bloom, 1997; and Riccio, Friedlander, and Freedman, 1994. 

is inqjortant to note that the studies of the programs in the education-focused category yield information about 
the effects of increasing welfare recipients’ participation in basic education programs (including Adult Basic Education, 
GED preparation, and English as a Second Language classes) and, to a much lesser extent, in vocational skills training 
programs, but not in college. On their own, many welfare recipients enroll in various types of education or training 
classes and reap benefits from them; the education-focused programs in the evaluation, however, sought to increase par- 
ticipation in education or training activities beyond what would normally occur. As will be discussed in Chapter 3, most 
of the programs did indeed increase such participation, but the increases in enrollments were in basic education courses 
and, to some degree, in vocational training courses, but generally not in college-level ones. 

"^See Scrivener and Walter, 2001, for a full discussion of the results of the direct test of Integrated versus Tradi- 
tional case management in the Columbus site. 
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as well as in the four other education-focused programs, almost all individuals were first assigned 
to either education or occupational skills training activities. 

This report also focuses on another program implementation dimension: mandatoriness. 
Past research suggests that the degree to which a program enforces a participation mandate for 
the welfare caseload is a determinant of whether a program can have an effect.^ High or low en- 
forcement of the mandate is a product of three factors: how wide a cross section of the welfare 
caseload is enrolled in a program; how closely a program monitors individuals’ participation; and 
how swiftly and consistently a program imposes financial sanctions (that is, reductions in 
monthly welfare grants) on those who do not participate. Nine of the programs were high en- 
forcement programs; Detroit and Oklahoma City were not, mostly because of limited program 
and staff resources. 

Table 1.1 categorizes the 11 programs according to their self-sufficiency approach and 
level of enforcement of the participation mandate. Section fV of this chapter discusses in greater 
detail these dimensions of the programs, as well as others that may have affected program im- 
pacts. It is important to keep in mind, however, that these program categorizations and descrip- 
tions apply to these particular programs as they would have been experienced by evaluation sam- 
ple members in their first three years of follow-up. In the years corresponding to sample mem- 
bers’ fourth and fifth years of follow-up, some of the programs changed their approach (in re- 
sponse to TANF). Section V of this chapter describes these changes. 

National Evaluation of Welfare-to-Work Strategies 
Table 1.1 

NEWWS Programs, Categorized by Approach, First Activity, 
and Enforcement Level 



Employment-] 


bcused approach 


Education-focused approach 


Job search first 


Varied first activity 


Education or training first 


High enforcement 


High enforcement 


High enforcement 


Low enforcement 


Atlanta LFA 
Grand Rapids LFA 
Riverside LFA 


Portland 


Atlanta HCD 
Grand Rapids HCD 
Riverside HCD 
Columbus Integrated 
Columbus Traditional 


Detroit 

Oklahoma City 



NOTES: LEA = Labor Force Attachment program. 
HCD = Human Capital Development program. 



’See Freedman et al., 2000a, p. ES-6; Hamilton and Scrivener, 1999, pp. 24-31; Bloom, 1997, p. 51; Kemple, 
Friedlander, and Fellerath, 1995, pp. ES-2 and ES-3; and Friedlander et al., 1987, pp. vii-x. 



II. Research Questions and Desigm 

Within the categorization scheme described above, the report analyzes program effects 
for single-parent welfare recipients and their children, focusing on results for the five years after 
individuals entered the programs. Data included in this report are from administrative records 
(unemployment insurance, state and county welfare payments, and Food Stamp payments) and 
from client surveys and parent and child assessments conducted over the five years after indi- 
viduals entered the study. The report specifically addresses the following questions: 

• Which welfare-to-work program approaches were most successful in helping 
welfare recipients to receive the program services or attain the skills or 
credentials that could enhance their chances of finding employment? 

• Which approaches were most successful in helping welfare recipients to find 
paid work? Did any approaches help individuals get “good” jobs, that is, full- 
time jobs with health benefits? 

• Which approaches were most successful in helping individuals to leave wel- 
fare and to remain off welfare? What were effects on Food Stamps as well? 

• Which approaches were most successful in increasing welfare recipients’ in- 
come and helping therh move out of poverty? 

• Which approaches were most successful in achieving self-sufficiency for those 
who lacked basic education credentials, were at high risk for long stays on 
welfare, or were at most disadvantage in finding jobs on their own? 

• Did any approaches affect the health care coverage of parents or their depend- 
ent children? 

• Did any approaches affect the household and personal circumstances of wel- 
fare recipients — for example, their marital status, family composition, or fer- 
tility, or the likelihood that they would experience abuse by an intimate part- 
ner? 

• Did any approaches affect the likelihood of using child care, the type of child 
care used, or the types of activities in which children participated on a typical 
day? 

• Did any approaches favorably or unfavorably affect the well-being of children 
of various ages as a result of the services provided to or the mandates imposed 
on parents? 

• In particular, did any approaches favorably or unfavorably affect the well- 
being of children who were preschool-age when their parents entered the study 
— children often viewed as most vulnerable to the effects of parents being out 
of the home? 



• Finally, which approaches were more expensive or less expensive? From the 
perspectives of government and the welfare recipients themselves, for which 
approaches did benefits outweigh the costs, and by how much? 

The NEWWS Evaluation uses an unusually rigorous research design, a random assign- 
ment experiment, to estimate program effects. In each site individuals who were required to par- 
ticipate in the program were assigned, by chance, to either a program group, which had access to 
employment and training services and whose members were required to participate in the pro- 
gram or risk a reduction in their monthly welfare grant, or a control group, whose members were 
not subject to a participation mandate and received no services through the program, but could 
seek out such services from the community.^ This random assignment design ensures that there 
are no systematic differences between the background characteristics of people in the program 
and control groups within each site when they enter the study. Thus, any subsequent differences 
in outcomes between the groups can be attributed with confidence to the programs. These differ- 
ences, called impacts, are the primary focus of this report. 

Although this design assures that the impact estimates of each program are extremely re- 
liable, there are some limitations. Local conditions, including labor markets, prevailing wages, 
welfare grant levels, political environments, program fimding levels, and staff administration, can 
all have an effect on the magnitude of impact estimates. For this reason, comparisons of impacts 
across the 1 1 employment- and education-focused programs in this report are only suggestive of 
the relative effectiveness of either approach in the short term.^ The most definitive judgments on 
the relative effectiveness of the two approaches come from the results in the three sites in this 
evaluation that tested versions of the two approaches side by side. 

Finally, it should be kept in mind that in some sites it was not possible to bar all control 
group members from receiving welfare-to-work program services for the entire five-year follow- 
up period examined in this report. While no control group members were exposed to the services 
provided and mandates imposed by sites’ welfare-to-work programs for the first three years of 
the follow-up period, their status differed by site in the fourth and fifth years, as detailed in Sec- 
tion VI of this chapter. As a result, in several sites impacts measured for years 4 and 5 of follow- 
up may underestimate the effects that would have been found had the control embargo been in 
effect for a full five years in all sites. Given this uncertainly, some of the analyses throughout the 
report separate impacts for years 1 to 3 from those for years 4 and 5. 

III. Program Environments 

When planning this evaluation, HHS and MDRC sought to include sites that would dem- 
onstrate operation in a diverse range of conditions, though they would not represent all welfare- 
to-work programs in the country. As shown in Table 1.2, sites varied along several dimensions. 



^Control group members were eligible for child care assistance similar to that offered to program group mem- 
bers if they were participating in nonprogram activities in which they had enrolled on their own. 

^While comparisons of impacts across programs or sites are not as reliable as the impacts for each program or 
site in the evaluation, they are a much more accurate determination of which types of programs are high and low 
performers than simple comparisons of statistics, such as welfare caseload reductions, across localities or states. 
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National Evaluation of Welfare-to-Work Strategies 
Table 1.2 

Program Environments 

Grand Oklahoma 

Characteristic Atlanta Rapids Riverside Columbus Detroit City Portland 
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1999 8,907 3,303 15,091 11,082 43,278 7,280 5,229 

Welfare grant level for a family of 3 ($) 

1993 280 474 624 341 459 324 460 

1998 280 459 6H 362 459 292 503 

(continued) 
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such as geographic location, labor market, and welfare grant level.^ More striking, however, are 
the changes in the economy, and the concomitant changes in welfare policy and caseloads, that 
all sites experienced from 1991, when the first group of individuals was randomly assigned as 
part of the NEWWS Evaluation, to 1999, the end of the five-year follow-up period for the last 
group of individuals randomly assigned. (Random assignment took place over a roughly two-year 
time period in each site, falling somewhere between mid 1991 and the end of 1994, depending on 
the site.) Part of the richness of the evaluation results thus stems from the examination of pro- 
gram effects over a period of time that reflected unprecedented economic growth and a sea 
change in welfare policies.^ 

To be included in the NEWWS Evaluation, sites needed large enough welfare caseloads 
to meet the sample size requirements of the research design. Accordingly, all seven sites include 
urban areas. Detroit, with a population that was slightly over 2 million in both 1990 and 1997, is 
the largest urban area studied in the evaluation. Riverside, with a population of over 1 million in 
1990, experienced the most growth during this time period, adding almost 24 percent to its popu- 
lation by 1997. Population growth in the other sites during this seven-year period ranged from 6 
to 14 percent. 

As population grew, so did labor markets. In three sites employment expanded signifi- 
cantly between 1991 and 1999: the employed labor force grew by 29 percent in Grand Rapids, 
28 percent in Riverside, and 25 percent in Atlanta. The other four sites experienced 12 to 18 
percent gains. 

Rising employment, particularly in localities with rising population, does not necessarily 
indicate declining unemployment rates. Unemployment rates in all seven sites, however, de- 
creased over this period. Following national trends, unemployment rates peaked in 1992 and, in 
general, were lowest in 1999. Early in the evaluation period unemployment rates in Detroit and 
Riverside topped 10 percent. Although rates in both localities steadily declined. Riverside rates 
remained at 8 percent in 1996, significantly higher than the national average. By 1999, unem- 
ployment rates in all evaluation sites were below 6 percent. Throughout the evaluation period the 
Columbus labor market was notably robust; its unemployment rate never exceeded 5 percent, 
even during the high point of the national recession, and dropped to less than 3 percent in 1999. 

Because individuals in the program and control groups within each site were subject to 
the same labor market, the quality of the economy by itself should not necessarily affect impact 
estimates; program and control groups shared the same advantages of a tight labor market or dis- 
advantages of a slack one. However, different economic environments can influence the type of 
people receiving welfare and thus required to participate in welfare-to-work programs For ex- 



®For a description of the site selection process, see Hamilton and Brock, 1994, Appendix A. 

The participation of single mothers in the labor market also increased dramatically during this time period. 
Rates of employment for single mothers with any children under age 18 increased from 57 percent in 1992 to 71 
percent in 1999 and for those with a child under age 3 increased from 35 to 56 percent over this same time period 
(U.S. House of Representatives, 2000, pp. 1412-1427). 

'“Data presented in this chapter are for the entire county (or counties) from which each site draws its sample 
members. 
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ample, in a good labor market individuals with more serious barriers to work are likely to be left 
on the welfare rolls. 

The size of welfare (AFDC or TANF) caseloads varied with the size of site populations, 
ranging in 1991, the beginning of the evaluation, from about 7,500 in Grand Rapids to almost 
90,000 in Detroit. In general, welfare caseloads grew in the early part of the evaluation period, 
peaked in 1993 or 1994, and declined to their 1991 levels or below by 1996, shrinking further to 
1999. In almost all evaluation sites, welfare caseloads in 1999 were no more than half the size 
they were in 1991; decreases in Riverside and Oklahoma City were somewhat smaller, with 
caseloads reduced by 35 and 41 percent, respectively. 

There was considerable variation in welfare grant levels among the sites. In 1993 monthly 
maximum cash payments for a family of three ranged from $280 in Atlanta to $624 in Riverside. 
Welfare grant levels in 1998 were similar to 1993 levels, with slight reductions in the maximum 
amount payable in three sites, a slight increase in two sites, and identical levels in two sites. Food 
Stamp payments, for which means standards are federally set, varied less across the sites, from 
$202 in Riverside to $292 in Atlanta, Columbus, and Oklahoma City in 1993, with slightly 
higher benefit levels in all sites in 1998." To some extent, low welfare grants are offset by higher 
Food Stamp payments, but this does not change the overall rankings of sites on benefit levels. 

All states were required to disregard (that is, not count) some earned income when calcu- 
lating a family’s welfare grant and, over time, more generous earnings disregards were put into 
place in some of the evaluation sites. At the beginning of the NEWWS Evaluation, five sites ap- 
plied standard earnings disregard rules. Under these, for the first four months of employment 
$120 of earnings and an additional one-third of the remainder were disregarded. This $120 disre- 
gard included both a $30 flat disregard and a $90 disregard for work expenses. In months 5-12 of 
employment, the additional one-third disregard was eliminated, leaving the total disregard at 
$120. After the first year of employment only the $90 work expenses disregard was allowed. In 
addition, individuals were allowed to disregard child care expenses up to $175 per child aged 2 
or over and $200 per child under age 2.*^ Atlanta and Riverside applied nonstandard disregard 
rules that permitted employed recipients to keep more of their welfare check. Throughout the 
evaluation period Atlanta employed “fill-the-gap” budgeting. Under fill-the-gap, working welfare 
recipients can earn up to the state-determined “standard of need” before losing all welfare bene- 
fits. For example, in 1993 the standard of need for a family of three was $424 (per month). A 
parent with two children could earn up to $756 in each of the first four months of employment 
and still remain on AFDC, $544 in months 5-12, and $514 per month thereafter. 

Throughout the course of the NEWWS Evaluation five-year follow-up period, states im- 
plemented further earnings disregard policies. In California, for example, the state received a 
waiver at the end of 1993 to eliminate the time limit on the standard earnings disregard applied to 
the calculation of welfare benefits and also instituted a version of fill-the-gap. By 1998, relative 
to 1993, all but one of the evaluation sites had implemented policies that allowed welfare recipi- 
ents to keep more of their earnings, affecting the likelihood that a sample member could work 



"These amounts assume the receipt of the maximiun welfare payment. 
'^Greenberg, 1992. 



while remaining on welfare. Increases were greatest in Riverside where, in 1998, a family of 
three could earn as much as $1,447 and still receive TANF. In Columbus and Oklahoma City, 
welfare recipients in 1998 similarly could earn almost double what they could in 1993 and re- 
main eligible for welfare. 

Differences in welfare grants, earnings disregard standards, and the use of fill-the-gap 
budgeting may explain some variation in program impacts, even though these grant levels and 
policies applied to both program and control group members in each site. Impacts on welfare 
payments in low-grant states are likely to be somewhat lower than those in high-grant states, 
other things being equal, because there are fewer welfare dollars to reduce. In addition, in low- 
grant states even low-paying jobs may be more attractive than welfare, providing a greater incen- 
tive to work. At the same time, in states that have higher grant levels, or generous earnings disre- 
gards, it may be easier for individuals to combine work and welfare in a way that will increase 
total household income and raise the family standard of living, particularly after factoring in the 
Earned Income Tax Credit (EITC).’^ 

IV. Program Features 

This report presents the long-term impacts, or effects, that the 1 1 NEWWS programs had 
on outcomes such as employment, earnings, welfare receipt, and child and family well-being. 
This section provides a context for interpreting the results in the chapters that follow by showing 
the range of programs on key implementation dimensions. As will become apparent, there is no 
typical “package” of welfare-to-work program features. As examples, the most work-focused 
programs are not necessarily the toughest, and those that use integrated case management do not 
necessarily monitor their enrollees’ progress more effectively than others. Given this information, 
it is important to interpret each program’s impacts as a result of its entire “bundle” of services 
and features. It is likely that a combination of features rather than one specific feature is associ- 
ated with successful outcomes and specific impacts. 

Nonetheless, this section focuses on the two implementation dimensions used to catego- 
rize the 1 1 NEWWS programs: the self-sufficiency approach used and the level of participation 
mandate enforcement. These particular dimensions are discussed at length here because they 
clearly demonstrate the division between the programs and provide a general framework for 
thinking about program results. In addition, this section explores other implementation features 
that provide an important context for interpreting the impact results in later chapters. 

It should be kept in mind that sample members were subject to the welfare-to-work pro- 
grams being studied only as long as they were receiving welfare. Welfare recipients frequently cycle 
on and off the welfare rolls. In the first two years of follow-up in these programs, for example, 
sample members received welfare and were required to participate in the programs for an average 
of 9 to 17 months, depending on the site. On average, sample members were either participating in 
a program activity, employed while subject to the program, or sanctioned for program nonparticipa- 



'^The federal EITC is a credit against federal income taxes for taxpayers with aimual earnings below a certain 



level. 



tion between 22 and 68 percent of these months, depending on the site.'"* (Those not in one of these 
statuses in any given month may have been waiting for case managers to refer them to an activity or 
sanction them, waiting for support services to be arranged or for activities to begin, temporarily de- 
ferred from participation for reasons such as illness or caring for an ill relative, or temporarily “lost 
track” of by program staff) While such figures on program “dosage” might seem surprising, it is 
important realize that the goal of welfare-to-work programs is to enable individuals to leave welfare 
and/or get a job. As a result, one would hope that sample members had not been participating in 
program activities during every month in the follow-up period, since this would mean that they had 
never left welfare and/or found employment during the period. 

As noted earlier, the implementation features discussed here primarily relate to how sam- 
ple members would have experienced the programs in the first three years of follow-up in the 
evaluation. Section V describes changes in program focus in the last years of the five-year fol- 
low-up period analyzed in the report. Table 1.3 summarizes for all programs the implementation 
features discussed in this section. 

A. Self-Sufficiency Approaches 

As discussed earlier, welfare-to-work program strategies usually emphasize either quick 
employment or an initial investment in education or training. The 1 1 programs in the NEWWS 
Evaluation blended elements of both strategies to varying degrees. 

The kinds of messages that case managers send about education and work, the emphasis 
that they place on different program activities, and the activities in which program enrollees ac- 
tually participate help to determine whether a client is more likely to get a job shortly after she 
enters the program or after she has tried to build her skills. The following program descriptions 
incorporate both the directions that case managers gave and the activities in which enrollees were 
most likely to participate.'^ Box 1.2 gives a brief description of the services offered by the pro- 
grams in this evaluation. 

Four of the programs are categorized as employment-focused and seven as education- 
focused. In the descriptions below, programs within each of the two categories are listed in rough 
rank order, from those that are most purely education- or employment- foe used to those that blend 
the two approaches. 

1. Education-Focused Programs 

The Oklahoma City program encouraged long-term education and training activities in- 
stead of active job search almost universally. Case managers communicated to clients the impor- 
tance of education, even in job clubs, as a way to increase skills for later entry into the labor market. 



'“'See Hamilton et al., 1997; Scrivener et al., 1998; Farrell, 2000; Storto et al., 2000; and Scrivener and Wal- 
ter, 2001. 

'^See also Hamilton et al., 1997; Scrivener et al., 1998; Farrell, 2000; Storto et al., 2000; and Scrivener and 
Walter, 2001. In addition, Bos et al., 2001, provides a detailed description of how adult education programs were 
implemented in the Atlanta, Grand Rapids, and Riverside HCD programs. 
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Oklahoma City Low High Selective Low Low Organizational emphasis No shortage Limited 

on providing assistance; 
licensed care only 

Portland High Moderate Moderately High Moderate Emphasis on necessity No shortage Strong 

selective of arrangements; choice of 



Box 1.2 



Structure and Content of Program Services* 

In general, the welfare-to-work programs studied in this evaluation made available 
to their enrollees the following services and classes: 

® Job club: Programs ran assisted job search activities, including classroom instruc- 
tion on techniques for resume preparation, job search, and interviewing, as well as 
a supervised “phone room” where participants could call prospective employers 
and search for job leads. Some sites employed job developers on staff, who 
searched for job leads in the community. 

® Basic education: This activity encompassed three different types of classes: Adult 
Basic Education (ABE) “brush-up” courses for individuals whose reading or math 
achievement levels were lower than those required for high school completion or 
General Educational Development (GED) classes; GED preparation and high 
school completion courses for individuals who did not have a high school diploma 
but wanted to earn one or its equivalent; and English as a Second Language (ESL) 
classes, which provided non-English speakers with instruction in spoken and writ- 
ten English. 

® Vocational training: Provided primarily through public schools, community col- 
leges, and Job Training Partnership Act (JTPA) agencies, these classes included 
occupational training in fields such as automotive maintenance and repair, nursing, 
clerical work, computer programming, and cosmetology. 

* College: Although this option was not used widely in the programs, some indi- 
viduals could attend college to fulfill their participation requirements. 

® Work experience: Participants could be assigned to three types of positions: un- 
paid work in the public or private sector (in exchange for their welfare grant), on- 
the-job training in the private sector, and paid work, usually in the form of college 
work study positions. 

o Child care and support services: All program participants, and control group mem- 
bers who enrolled in activities on their own, could be reimbursed for child care costs 
incurred as a result of participation. Also, if they met the eligibility criteria, all 
program and control group sample members could be reimbursed for child care ex- 
penses incurred while they were employed and no longer receiving cash assistance 
through the federal transitional child care (TCC) program. Funds were also available 
for work-related expenses, such as uniforms or books, and for transportation costs, 
such as public transportation passes or per-mile automobile reimbursement. 



♦For a more detailed description of service components in the 1 1 programs, see Hamilton et al., 1997; Scriv- 
ener et al., 1998; Farrell, 2000; Storto et al., 2000; and Scrivener and Walter, 2001. 
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The Atlanta HCD, Grand Rapids HCD, and Columbus Integrated and Traditional 
programs emphasized increasing skills through formal education and training before entry into 
the labor market. Because of the generally low educational attainment of participants in these 
programs, basic education was a common first activity, though Grand Rapids also encouraged 
participation in vocational training programs. Clients in these programs were given considerable 
latitude in choosing the kind of education activity they wanted to pursue. 

The Detroit program underwent a substantial shift in focus over the study period. Ini- 
tially, the program emphasized long-term education and training assignments before Clients en- 
gaged in work search. About midway through the study period clients were referred to a program 
that required job search first. 

The Riverside HCD program, which enrolled only individuals without a high school di- 
ploma or GED, generally assigned clients to basic education as a first activity. Short stays in 
these classes and active job search once a literacy benchmark was reached were stressed by case 
managers throughout clients’ participation. Job developers assisted HCD clients in job club. 

2. Employment-Focused Programs 

Case managers and program staff in the Riverside, Grand Rapids, and Atlanta LFA 
programs emphasized that employment was the goal of program participation and that job 
search should be the first activity for participants. Clients were given very little choice in their 
first program assignment. In Riverside participants were encouraged to take even part-time and 
low-paying jobs as a first step to self-sufficiency and were assisted by full-time job developers 
who searched for job leads and followed up on job placements.'® While Grand Rapids staff 
stressed to clients the importance of finding work, they believed that it might be justifiable for 
clients to turn down temporary or part-time jobs. Those who wished to enroll in education pro- 
grams were encouraged to do so — in addition to, not instead of, working. Atlanta case managers 
indicated the availability of education and training services as a second step after initial job 
search. Many Atlanta enrollees did, in fact, participate in education or training if they completed 
job search without finding a job. 

While Portland program staff emphasized that employment was the goal of program 
participation, not all enrollees were assigned to job search first. For individuals who first enrolled 
in education or training activities, usually those who were thought by case managers to be the 
more disadvantaged members of the caseload, program staff communicated that improving em- 
ployability was the goal of their assignment. Portland also employed full-time job developers to 
work with participants once they began actively looking for a job though, unlike other developers 
in work- focused programs in this evaluation, they encouraged participants to seek “good” jobs, 
that is, higher-paying jobs with benefits. 



'®Given Riverside’s high grant level and earnings disregard policies, individuals could remain on welfare until 
they had a full-time wage of $8.35 per hour. 



B. Degree of Participation Mandate Enforcement 

In addition to the messages about work and education that case managers send to clients 
and the relative mix of services that a program provides, the degree to which a program enforces 
a participation mandate has also been shown to affect program impacts. The three elements of 
enforcement include the broadness with which a program enrolls from its caseload, how well it 
monitors participants’ progress, and how strictly the participation requirements are enforced. In 
other words, a high or low ranking indicates the likelihood that a client would be told to partici- 
pate, the likelihood that her case manager would know if she had not been participating, and how 
swiftly or surely she would be sanctioned for not participating. Nine of the 11 NEWWS pro- 
grams were “high enforcement” programs; that is, they were rated the equivalent of “high” on at 
least two of the three elements. Two programs, Detroit and Oklahoma City, were rated the 
equivalent of “low” on all three elements of enforcement. (See Table 1.3.) 

The rest of this section describes how each program was rated with respect to the en- 
forcement of the participation mandate. Within each element of enforcement sites are listed in a 
rough rank order, from high to low. A number of factors can contribute to a program’s overall 
ranking on an element; a site may be high on one but low on others, but these factors have not 
been formally \yeighted. 

1. Broadness of Enrollment 

How likely was it that an individual would have been required to participate in the 
welfare-to-work programs?*’ 

Single parents with children aged 3 or over were required to participate in all programs 
studied in this evaluation, with some programs requiring participation of women with children as 
young as age 1. Individuals who had health barriers or were taking care of a household member 
who was ill or incapacitated, were pregnant, or were already working 30 hours per week could be 
exempted from this mandate. ' ^ 

At a number of points, administrator and case manager discretion, combined with fund- 
ing and resource constraints, could affect a welfare applicant’s or recipient’s chances of enroll- 
ing in a welfare-to-work program. First, five of the programs required women with children as 
young as age 1 to enroll. Since over 40 percent of the welfare cases nationwide in 1994 in- 
cluded a child under age 3,'^ expanding the mandate to this group significantly increased the 
proportion of the caseload that could be served by the program. Second, case managers might 
not tell all of those who meet the demographic criteria to enroll. Third, individuals might not 
show up for the program orientation because they do not wish to participate or they become 
exempt or leave welfare in the period between referral and orientation date, especially if the 



’’See also Hamilton and Brock, 1994, pp. 51-55, for a more detailed description of the sites’ enrollment practices. 

Exceptions for health reasons were typically few. In Columbus, for example, where detailed data on exemp- 
tions are available, the majority of exemptions were due to the youngest child being under age 3; less than 9 percent 
of the exempt single parents had a long-term illness or incapacitation and less than 3 percent were caring for an ill or 
incapacitated family member (including a child). See Hamilton, 1995. 

'^.S. House of Representatives, 1996, Table 8-32; percentage of all AFDC households with a child under 
age 3. 
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period is long.^° Finally, even recipients who attend an orientation could be deferred from fu- 
ture activities at case managers’ discretion.^’ 

The Grand Rapids LFA and HCD, Columbus Integrated and Traditional, and Riv- 
erside LFA and HCD programs enrolled broadly, including virtually their entire mandatory 
caseload.^^ Both of the Grand Rapids programs included parents with children as young as age 1 
in their participation mandate. 

The Atlanta LFA and HCD programs aimed to enroll their entire mandatory caseloads; 
however, budget limitations created a waiting list, sometimes as long as six months, before those 
who had been referred to the program could actually enroll. During a waiting period welfare re- 
cipients with the fewest barriers to work leave the rolls on their own; thus, the clients who actu- 
ally enroll may be slightly more disadvantaged than they would be if there were no waiting list. 
Indeed, the Atlanta sample includes more long-term recipients than samples in most other sites. 
Because the Atlanta programs did refer virtually all members of their mandatory population to 
the program, and enrolled all those who were left after the delay, their enrollment is termed 
“broad-delayed” in Table 1.3. 

The Portland program extended its mandate to parents of very young children (as young 
as age 1), but selectively enrolled from its mandatory population. Some individuals determined 
“hard-to-serve,” that is, less employable, either would not be referred for enrollment in the pro- 
gram or, after attending a program orientation, would not be assigned to further activities. For 
these reasons Portland can be considered moderately selective. 

The Detroit and Oklahoma City programs also extended their mandate to women with 
very young children, but were more selective than other programs. Like Atlanta, Detroit had a 
waiting list for “slots” in the program. Guided by the principle that the program would rather 
spend scarce resources on those who wished to participate than on cajoling those who might 
never participate, staff tended to give priority to “mandatory” clients who volunteered for the 
program. In addition, case managers spent a large proportion of their time authorizing child care 
and support service payments, leaving little time to focus on individuals who were not eager to 
enroll. Oklahoma City referred all those eligible to its program; however, since the program was 
also limited by resources and rising caseloads, much of the responsibility for enrolling in pro- 
gram activities fell on the client. Case managers assisted clients in finding appropriate services, 
but the self-directed enrollment allowed resistant individuals to avoid the mandate. As discussed 
in Chapter 2, this evaluation examines only the experiences of individuals applying for welfare 



“See Hamilton and Brock, 1994, and Knab et al., 2001, for a discussion of the length of time between referral to 
and enrollment in welfare-to-work programs and reasons for orientation nonattendance. 

^*As described in more detail in Chapter 2, in most of the sites, random assignment to research groups occurred 
once individuals attended program orientations. Only in Columbus and Oklahoma City did random assignment occur 
earlier. 

^^In Riverside, individuals working 30 hours per week or more were not enrolled in the program. In addition, 
program enrollees who were employed 15 to 29 hours per week were not assigned to additional, concurrent program 
activities. 

^^See Bane and Ellwood, 1983; Pavetti, 1992; Gueron and Pauly, 1991; and Hamilton and Brock, 1994. 
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(“applicants”) in the Oklahoma City program; the treatment of those already receiving welfare 
(“recipients”) may have been different from the situation described here. 

2. Closeness of Participation Monitoring 

How often or how quickly would an enrollee be contacted by her case manager if 
she was not participating? 

Once clients begin participating, they may drop out of activities or attend irregularly 
because they have a new job, have new problems with child care or transportation, or no longer 
want to participate. Close monitoring can help case managers maintain and increase participa- 
tion among their caseload, facilitate the authorization of transitional benefits for individuals 
who leave welfare for work, or speed case closures for individuals who become ineligible. In 
order to monitor participation closely, case managers must learn about attendance problems 
from activity providers, determine the reasons for them, contact clients about their options or 
the consequences of nonparticipation, and then inform the income maintenance branch of a 
case’s outcome. How closely an individual will be monitored depends on the level of informa- 
tion that case managers get from the activity instructors and providers and on the time that case 
managers have to devote to this task. 

The Riverside LFA and HCD, Grand Rapids LFA and HCD, and Portland pro- 
grams all intensively monitored their participants’ progress. Overall, more case managers in 
these sites indicated receiving a lot more information about attendance from providers than those 
in most other sites. In addition, case managers reported that it took them between one and two 
weeks to both hear about attendance problems from providers and contact clients about their at- 
tendance, the shortest in the range of time among the programs. 

The Atlanta LFA and HCD and Columbus Integrated programs engaged in moderate 
monitoring of their clients. Information sharing between providers and case managers was not as 
regular in these programs as in the intensive-monitoring programs, and it took between two and a 
half and three and a half weeks to get information from providers. These programs did, however, 
contact clients in less than two weeks once they learned of attendance problems. 

The Oklahoma City, Detroit, and Columbus Traditional programs engaged in less in- 
tensive monitoring of their clients than the other programs. Regular protocols for obtaining at- 
tendance information from providers were not in place for at least two of the programs. It took a 
little longer, on average, for all three programs to get information from providers than it did for 
the moderate-monitoring programs. Moreover, it took between two and three weeks for case 
managers to contact clients about their attendance problems, on average one week longer than for 
the moderate-monitoring programs. 

3. Level of Mandatoriness 

How much would an individual be encouraged, or coerced through financial sanc- 
tions, to participate in a program activity if she did not want to? 

The great majority of welfare recipients who are required to participate in welfare-to- 
work programs believe, prior to hearing details about the program, that they will have trouble 
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participating, citing barriers such as a lack of child care or transportation or having a health or 
emotional problem.^'* All the programs in this evaluation provided monetary assistance to help 
participants (both program and control group members) with child care and transportation, but 
they also relied on case managers to work with clients to remove participation barriers or to co- 
erce participation through the imposition of a financial sanction.^^ Most of the programs were 
strongly committed to enforcing the participation mandate for their welfare caseload, though the 
degree to which clients were more likely to be cajoled or coerced differed. Individuals in Detroit 
and Oklahoma City were not as likely to be cajoled into participating if they did not want to, 
though this was largely the consequence of limited program funding and staffing. 

The Grand Rapids LFA and HCD and Columbus Integrated and Traditional pro- 
grams were very highly committed to the enforcement of clients’ participation obligation. Case 
managers sent strong messages about the consequences of nonparticipation and, in instances of 
noncompliance, imposed financial sanctions swiftly on a large percentage of their caseloads. 

While other programs informed clients of the necessity of program participation, they 
gave them more chances to comply than Grand Rapids or Columbus. Atlanta LFA and HCD 
program case managers were somewhat less comfortable with enforcing participation require- 
ments through financial sanctions, though they did so on a regular basis. More clients were sanc- 
tioned in the Atlanta HCD program than in its LFA program, though the messages that case man- 
agers sent about requirements were not different. 

Riverside LFA and HCD program staff tended to view sanctioris as one tool to get cli- 
ents to attend activities and initially emphasized to clients the importance of personal responsibil- 
ity. Riverside staff did not delay requests for or impositions of sanctions; the process, however, 
took longer than it did in most other programs because of extensive state-mandated due process 
procedures. The Portland program staff also emphasized ways to solve problems related to 
nonparticipation rather than reductions in clients’ grants. Staff in Riverside and, to a greater ex- 
tent, in Portland were more willing to defer individuals from participation requirements than staff 
in either Columbus or Grand Rapids. Staff in Portland did, however, ultimately sanction non- 
compliant individuals. 

The mandatory participation requirements in the Detroit and Oklahoma City programs 
were communicated less intensively to clients. As already mentioned, staff in these two sites fo- 



^“'See Hamilton and Brock, 1994. 

^^The financial sanctions in effect during the NEWWS Evaluation were not the “full-family” sanctions currently 
being implemented in numerous states. During the period under study, sanctions affected only the adult member of 
the welfare case; for a three-person family in 1993, for example, a sanction would reduce the welfare grant by ap- 
proximately 20 percent, depending on the site. In addition, the financial penalty continued until the sanctioned indi- 
vidual complied with the program participation mandate, with a minimum sanction length of three months for the 
second “offense” and a minimum length of six months for the third offense (with no minimum length for the first 
offense). As a result, some sanctioned individuals endured a penalty for a short amount of time while some experi- 
enced a penalty for much longer. For example, almost half of those sanctioned in the two Grand Rapids programs 
were in this status for at least 12 months of a 24-month follow-up period. See Hamilton et al., 1997; Scrivener et al., 
1998; Farrell, 2000; Storto et al., 2000; and Scrivener and Walter, 2001, for more details on sanction practices in 
each of the NEWWS Evaluation sites. 
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cused on those who wanted to participate. Resource constraints kept staff from following up on 
nonparticipation, and staff tended to delay imposing sanctions. 

C. Other Program Features 

1. Child Care Supports for Participation and Work 

How much support in the form of child care assistance could an individual expect 
for her participation in a program or subsequent employment? 

For many welfare recipients with young children the major obstacle to working or attend- 
ing an education or job training program is child care. All 1 1 programs studied in the evaluation 
provided this assistance to participants in the program (and to control group members who en- 
rolled in activities on their own in the community) as well as transitional child care (ICC) for 
those who left welfare for work. However, the relative emphasis that the programs placed on 
making this assistance available and the messages that case managers sent to clients about the 
type of care they should choose varied by site {not by research group within site). 

Participation-related child care. In the Atlanta LFA and HCD, Oklahoma, Portland, 
and Detroit programs, child care assistance was emphasized either by site staff or by the wel- 
fare department’s organizational structure. In both Atlanta programs case managers actively pro- 
moted the availability of child care reimbursement as a benefit of program participation and even 
used it as an inducement for noncompliant clients to participate. In Oklahoma statewide empha- 
sis on access to child care made assistance to clients readily available while they were in the pro- 
gram and after they left welfare for work. Oklahoma had no set caps on the amount of child care 
assistance that clients could receive. Atlanta and Oklahoma reimbursed only for care given by 
licensed providers. 

Portland program caseworkers told clients that not having child care arrangements was 
not an acceptable reason for not participating in program activities. Staff often encouraged clients 
to have backup arrangements in case their regular provider fell through. Although case managers 
did not push specific types or locations of providers, they did emphasize the necessity for clients 
to make arrangements and assisted clients who were unable to make arrangements on their own. 

Detroit program case managers reported that they spent much of their time on child care 
payment authorizations and that the priority placed on making child care payments took time 
away from employment and training counseling. Detroit staff would make referrals to licensed 
providers in the area on request, but the choice of provider (including choosing licensed child 
care or unlicensed care approved by the welfare department) was left to the client. 

Both the Grand Rapids and Columbus programs would reimburse expenses from child 
care in licensed as well as unlicensed care, but expected clients to make their own arrangements. 
Referrals to licensed providers in the area could be made for clients at their request. 

In all sites except Riverside, case managers said that child care providers were not diffi- 
cult to come by. In Riverside, case managers noted that some area providers did not like working 
with the program or its participants because they did not approve of the reimbursement rates or 
procedures. These case managers encouraged clients to use low-cost, more informal arrange- 
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ments, both to contain program costs and because they believed that clients would be more able 
to afford such arrangements after program or other government supports expired. Clients and 
case managers often clashed about the providers they wished to use, especially if clients chose 
more expensive care. 

Transitional child care. In the Detroit, Portland, Columbus, and Oklahoma City pro- 
grams, authorization for TCC payments did not appear to be difficult. In Portland, Columbus 
Integrated, and Oklahoma the use of integrated case managers, who are more likely to know both 
the welfare and employment information needed to determine if a client is eligible for TCC, may 
have made authorization in these sites easier for both program and control group members. 

In the Atlanta, Grand Rapids, and Riverside LFA and HCD programs, few clients 
who began working received TCC; case managers in all three sites cited a lack of information 
about clients’ welfare status when authorizing child care payments. Thus, TCC authorization in 
these sites would have been infrequent for both program and control group members. 

2. Culture of Eligibility to Culture of Self-Sufficiency: Integrated 
Case Management 

How likely was a welfare recipient to get a unified self-sufficiency message from the 
welfare department? 

The eligibility-compliance culture of the welfare system (more common prior to FSA), in 
which contact between a client and an agency is focused solely on determining eligibility for 
staying on welfare, has been harshly criticized. Implementing a mandatory welfare-to-work pro- 
gram was one way that welfare offices hoped to change from an eligibility-compliance culture to 
a self-sufficiency culture, which would structure interactions and expectations around leaving 
welfare for work and preparation and supports for it. Yet this task is formidable; it requires the 
income maintenance and employment services staffs of the welfare offices to work together to 
send a unified message of the self-sufficiency goal to the client. If the sole responsibility for de- 
livering the self-sufficiency message is remanded to the employment and training program, pro- 
grams can be interpreted by clients and workers as requirements for continued receipt of assis- 
tance, or another element of compliance, instead of an overhaul of the philosophy of the welfare 
department. Implementing an integrated case management approach, in which one worker is re- 
sponsible for both the eligibility determination and employment services functions, is one way 
that has been suggested to achieve a more unified culture.^® Three of the programs in the 
NEWWS Evaluation used integrated case management, but these and the other eight programs 
met with different levels of success in coordinating the messages between their eligibility and 
employment preparation staffs and in refocusing the welfare department’s interactions with cli- 
ents on the road toward self-sufficiency. 

As part of a specially formulated research experiment, the Columbus Integrated pro- 
gram used integrated case management. Staff had sufficient resources and small enough 
caseloads that they were able to perform both their income maintenance and self-sufficiency 
roles. Thus, an individual’s case manager could both monitor her progress in becoming self- 



^*See Scrivener and Walter, 2001, pp. 3-5; and Bane and Ellw^ood, 1994, p. 127. 



sufficient and verify her credentials for staying on welfare. As of the two-year follow-up point, 
this program had the largest effect on changing clients’ minds about whether they agreed that the 
welfare office tried hard to get recipients employed or enrolled in school. 

The Portland program was marked by a strong partnership between welfare-to-work 
staff (eligibility workers and integrated case managers) and case management staff contracted by 
the Portland welfare department. The division of labor was flexible between contractor staff and 
welfare department integrated case managers, with responsibility for case management services 
such as reassignment to activities and attendance monitoring, as well as a mission of promoting 
self-sufficiency, shared by both. Moreover, eligibility workers in Portland were among the most 
knowledgeable about the program and spent more time discussing the program with recipients 
than those in most other programs. These results suggest that together eligibility workers, inte- 
grated case managers, and contractor staff were able to send a unified self-sufficiency message to 
welfare recipients. 

The Oklahoma City program also used integrated case management. However, limited 
resources and large caseloads led case managers to put little overall emphasis on the employment 
services function of their position; in fact, their performance evaluation benchmarks were primar- 
ily related to the accuracy of their eligibility duties. Like Portland, Oklahoma City supplemented 
its integrated case managers with some caseworkers who focused on employment-related ser- 
vices. However, owing to staffing constraints, not all clients received this added case manage- 
ment. The result was a program with little overall emphasis on self-sufficiency. 

The Atlanta, Grand Rapids, Riverside, and Columbus Traditional programs all used 
a separated, or “traditional,” case management structure, in which a client had two different case 
managers, one who specialized in determining eligibility and processing payments and another 
who focused on her participation and progress in a welfare-to-work program. Although the dif- 
ferent staffs did not report any major problems in their working relationship, they mentioned that 
there was a lack of partnership between the two. Income maintenance workers knew little about 
the programs and most often discussed with clients the penalties for nonparticipation in the pro- 
gram, not the services it provided, suggesting that participation was cast as a compliance re- 
quirement and not a route to self-sufficiency. 

The separation between the two staffs of the welfare department in the Detroit program 
was even more pronounced. Income maintenance workers knew little about the program and had 
almost no contact with clients regarding their participation; the welfare-to-work program case 
managers in Detroit handled some income-related functions related to program participation, 
such as child care payments, that income maintenance workers were responsible for in the other 
traditional sites. Staff mentioned that this separation was intentional, so that the welfare-to-work 
case managers would be able to communicate consistent messages and information. In short, the 
priorities of the two staffs were so dissimilar that an individual was likely to experience very dif- 
ferent cultures during her contact with the department. 
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V. Program Changes in the Last Two Years of the 
Five-Year Follow-Up Period 

The previous section described the 1 1 programs as they existed during the first three years 
of the follow-up period. In the last two years of the five-year follow-up period, however, the focus 
of many of the programs changed. Most commonly, some of the education-focused programs be- 
came more employment- focused, a change driven in part by the 1996 welfare reform law.^^ This 
shift was not incompatible with what might normally happen in an education-focused program. It 
would have occurred several years after most program group members started their participation in 
education or training activities. Most education-focused welfare-to-work programs involve periodic 
job search, particularly once individuals have improved their skills or achieved a credential. There- 
fore, it is unlikely that the shift to an employment-focused program would have greatly compro- 
mised the nature of the treatment studied in these originally education-focused programs. In addi- 
tion, the changes in program emphases would have affected only those program group members 
still receiving welfare in the last two years of the five-year follow-up period. As discussed in the 
next section, which covers changes in control group members’ treatment over time, this proportion 
varied by site. The following paragraphs highlight the most important program changes. 

In Atlanta, the site (and state) implemented a strongly job search- focused “work first” 
program in December 1996. For LFA sample members, the new program continued their em- 
ployment-focused treatment. For HCD sample members, the new program represented a change 
in program message and emphasized activities. At the same time, Georgia began the count of 
months toward a welfare time limit (which would have affected sample members in all three of 
the Atlanta research groups). 

In Grand Rapids, a Work First program was implemented during the five-year follow-up 
period as well, in October 1994, but referral to this program was delayed for most LFA and HCD 
sample members. For individuals assigned to the LFA and HCD research groups in roughly the 
first half of the Grand Rapids random assignment period, referrals to the Work First program 
could be made three years after random assignment if these sample members were still receiving 
welfare. Reviews of Work First program databases, however, indicated that as of May 1996, 
which would have been about the fourth year of the five-year NEWWS follow-up period for 
most of these sample members, less than 10 percent of this group had, in fact, been referred to 
the new program. Individuals assigned to the LFA and HCD research groups in roughly the sec- 
ond half of the Grand Rapids random assignment period continued to be part of the original LFA 
and HCD programs throughout the five-year follow-up period. Grand Rapids also took part in 
another statewide initiative. Project Zero, which sought to drastically reduce the number of non- 
working adults on the welfare caseload, but implementation of this program did not occur until 
almost all NEWWS sample members were beyond the end of their five-year follow-up period. 



^^Three evaluation sites also implemented welfare time limits during the later part of the five-year follow-up pe- 
riod. These would have applied to both program and control group members. This means that the count of months 
toward their welfare limit would have started; no sample members actually would have reached their welfare time 
limit during the follow-up for this evaluation. 
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In Riverside, the LFA program focus and components generally carried over into years 
4 and 5 of the five-year follow-up period. The HCD program, however, had more of an em- 
ployment focus in years 4 and 5. Once HCD sample members completed their education activi- 
ties in the last two years, it is likely that they would have been assigned, if still on welfare, to 
job search. If they did not find a job through job search, it is unlikely that they would have 
been assigned to more basic education; rather, they probably would have been assigned to do 
more job search or to attend group self-esteem-building sessions. Assignments to vocational 
training remained rare for both LFA and HCD sample members in the last two years of follow- 
up, as it had in the first three years. (The CalWorks program in California was implemented at 
a point that would have been, for most Riverside sample members, after the end of the five- 
year follow-up period examined in this report; its provisions are thus largely not reflected in 
the behavior of sample members examined here.) 

In Columbus, several program changes took place in October 1997. First, all sample 
members in both the Integrated and Traditional program groups began to receive integrated case 
management, reflecting a county-wide shift to this type of arrangement. Second, the program be- 
came much more employment-focused than education-focused. Third, Ohio began the count of 
months toward a welfare time limit (which would have affected sample members in all three Co- 
lumbus research groups). 

Reflecting the Michigan changes outlined above for the Grand Rapids site, the Detroit 
program was transformed in October 1994 to the strongly employment-focused Work First pro- 
gram, operated by an agency other than the welfare department. In this site, unlike Grand Rapids, 
it was not possible to implement procedures whereby program group members could continue to 
be eligible for their “original” program. As a result, sample members still on welfare in late 1994 
would have experienced an abrupt change in program type and focus, from an almost voluntary 
program emphasizing education to a strongly mandatory one emphasizing employment. 

The Oklahoma City program also became more employment-focused in years 4 and 5 of 
the five-year follow-up period. In addition, in October 1996 Oklahoma began the count of 
months toward a welfare time limit (which would have affected both program and control group 
members in the site). 

In Portland, program group members would have continued to receive an employment- 
focused treatment in the last two years of the five-year follow-up period. Several other program 
changes, however, occurred in these years. First, over the course of the five years, the program 
made more use of job search and less use of education. In the last year of follow-up, for example, 
program staff were urged to reduce the number of assignments to GED classes and, when made, 
to limit the assignments to only three to six months. In 1999, GED and other education programs 
were viewed more as activities that could help working individuals retain and advance in jobs 
than as activities for initial work preparation. Along with this, life skills activities were still of- 
fered, but were compressed to two weeks, from the five to six weeks observed in the first three 
years of the follow-up. Second, in the last two years of follow-up, staff made more short-term 
training assignments (of three to six months) and worksite placements (a type of “supportive” 



^*See Farrell, 2000, for details on how this change came about. 
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work experience). Finally, toward the end of the follow-up period, Portland started to move away 
from a type of integrated case management and toward traditional case management. 



VI. Control Group Treatment Over Time 

Part of the reliability of the findings from a random assignment social experiment rests 
on the assumption that control group members are not exposed to the specific program being 
evaluated. If control group members are exposed to the program, measured impacts could rep- 
resent an underestimate of the true effects of the program. Rarely, however, is it possible to 
maintain this lack of exposure for a long period of time. In this regard, the NEWWS Evalua- 
tion was no exception. 

Agreements reached with the NEWWS Evaluation sites at the beginning of the study 
specified that all control group members were to be kept out of the welfare-to-work programs 
being studied in each site for at least three years from their random assignment date. (As noted 
earlier, control group members were free to enroll in other employment-related activities offered 
in their communities during this period and, as will be discussed below and in Chapter 3 of this 
report, many controls did, in fact, enroll in such activities.) Midway through the evaluation, pro- 
gram operators in Riverside and MDRC agreed to extend the embargo on providing welfare-to- 
work program services to control group members to five years. Program operators in Portland 
and Grand Rapids agreed to a similar extension, but only for a subsample of control group mem- 
bers. In Portland, about one-quarter of the control group was randomly selected (from every 
month of the random assignment period) to remain ineligible for welfare-to-work program ser- 
vices until the end of year 5; in Grand Rapids, the embargo on welfare-to-work program services 
was extended for all control group members who were randomly assigned during 1993, the last 
year of sample intake in the site. However, Atlanta, Columbus, and Oklahoma City could not 
continue barring control group members from welfare-to-work program services longer than 
three years because of the implementation of welfare time limits in their states; if NEWWS con- 
trol group members could not be exempted from welfare time limits, then both HHS and MDRC 
felt it necessary to allow control group members to be eligible for some type of welfare-to-work 
program services once their time limit clock “starting ticking.” Finally, in Detroit, it was not fea- 
sible to continue the control services embargo, after a different agency took over administration 
of the site’s welfare-to-work program and entry points into the new program were so numerous 
and widespread that screening to identify NEWWS control group members would have been 
close to impossible. 

Most probably program impacts on employment and earnings and other outcomes in the last 
two years of follow-up in a few of the five sites above would have been somewhat larger had some 
control group members not been exposed to welfare-to-work programs. For several reasons, how- 
ever, lifting the control group embargo on services prior to the end of the five-year follow-up period 
in these five sites most likely had only a small effect on measured program impacts. As discussed in 
detail in Chapter 2, most control group members were not eligible to receive program services 

^*The authors know of only one large-scale social program evaluation — the GAIN Evaluation, a study of Cali- 
fornia’s late 1980s and early 1990s welfare-to-work program — that, by design, kept control group members from 
exposure to the specific program being evaluated for a follow-up period as long as five years. 
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when the control group embargo ended, often because they had already left welfare. From one- 
quarter to one-half of control group members in these five sites were receiving welfare when their 
embargo on program services was lifted. In addition, after the embargo was lifted, some control 
group members were not contacted about enrolling in the program until after the end of the follow- 
up period, and others were assigned to a program orientation but did not show up. Moreover, the 
likely effects of ending the control group embargo were estimated for the sites where the embargo 
was lifted by calculating impacts for a subsample of control group members who were precluded 
from program services for four to five years. It was found that the patterns of impacts in years 4 and 
5 resembled those for all sample members in those sites. 

It should be emphasized that the control group situations described above do not affect 
the assessments in this report of the relative merits of the Labor Force Attachment and Human 
Capital Development approaches in welfare-to-work programs. As described in Chapter 2, the 
three-group random assignment designs in the three sites in which these two types of programs 
were simultaneously operated permit a direct comparison of these two approaches, that is, a 
comparison that does not need to take into account the services received by and the behavior of 
control group members. 

VII. A Brief Review of Two-Year Impact Results 

Several reports have documented the range of effects achieved by all 1 1 NEWWS pro- 
grams within a two-year follow-up period.^® This report extends and expands on these two-year 
findings, lengthening the follow-up period to five years and including additional outcome meas- 
ures. As a basis for understanding the long-term findings, this section recaps the two-year find- 
ings, noting which welfare-to-work program strategies were more or less successful in helping 
welfare recipients achieve self-sufficiency in the relatively short run. 

All programs, regardless of their approach, increased participation in activities de- 
signed to promote employment during the two-year follow-up period. As expected, employ- 
ment-focused programs increased participation primarily in job search activities, whereas educa- 
tion-focused programs increased levels primarily in basic education and vocational skills training 
classes. Very different patterns of participation impacts were found for individuals who entered 
the study with a high school diploma or GED certificate and for those who did not have these 
credentials. In most education-focused programs participation impacts were concentrated among 
those without a high school diploma or GED and resulted primarily from large increases in atten- 
dance in basic education; only small increases in attendance in post-secondary education or voca- 
tional training were found for the education-focused programs, and they were generally among 
only high school graduates or GED holders. In contrast, large impacts on participation in job 
search were achieved for both groups in the employment-focused programs. 

Some education-focused programs, as well as the Portland program, were able to 
produce relatively large two-year impacts (about 10 percentage points) on GED attainment 
among sample members who did not have a high school diploma or GED certificate at 
study entry. Of the seven education-focused programs, the Grand Rapids HCD, Riverside HCD, 

^®See Freedman et al., 2000a; McGroder et al., 2000; and Hamilton, 2000. 
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and Columbus Traditional programs had this effect. The Portland program, in addition to boost- 
ing GED receipt, increased the rate at which those without education credentials obtained a trade 
license or certificate by 12 percentage points. For sample members with a high school diploma or 
GED certificate at study entry, only three programs (Atlanta LFA and HCD and Grand Rapids 
HCD) increased receipt of a trade license or certificate. 

As expected, employment-focused programs produced larger gains in employment 
and earnings over the two-year follow-up period than education-focused programs. A ma- 
jority of control group members in all sites except Riverside, the site with the most difficult labor 
market, found jobs on their own at some point within two years of random assignment and, as a 
group (including zeroes for noneamers), had average earnings during the second year of follow- 
up ranging from $2,127 (Oklahoma City) to $3,978 (Columbus). The Portland program attained 
the largest earnings increase of all programs, with members averaging more than $900 per year in 
earnings above control group members. Equally important, employment and earnings gains in 
Portland grew larger over time and reached their highest levels at the end of year 2. The other 
employment-focused programs produced moderate earnings increases, ranging fi-om $400 to 
$650 per year, that grew smaller toward the end of year 2. 

Several of the education-focused programs began to show moderate impacts in year 

2. By the end of year 2 all but two of the education-focused programs had attained increases in 
employment and earnings that equaled or exceeded the gains achieved by all employment- 
focused programs except the Portland program. The two exceptions to this pattern, the Riverside 
HCD and Oklahoma City programs, did not raise employment or earnings levels in year 2. 

All programs reduced two-year welfare dependency to some degree. Control group 
members in all but one site remained on welfare for an average of 16 to 20 months during the 
two-year follow-up period and received payments averaging between $3,624 (Oklahoma City) 
and $10,302 (Riverside HCD) during this period. Seven of the 11 programs, a mixture of em- 
ployment- and education- focused approaches, decreased cumulative welfare expenditures by 
more than 10 percent, a historically large effect; welfare reductions in the other four programs 
were smaller. The Portland program produced a large decrease in welfare receipt that persisted at 
a high level throughout the follow-up period, showing a 12 percentage point decrease in welfare 
receipt during the last quarter of the two-year period; all other programs had reduced welfare re- 
ceipt at this point by 3 to 7 percentage points. However, at least 40 percent of sample members in 
the programs were still relying to some extent on welfare at the end of two years. 

Most programs increased sample members’ reliance on earnings rather than wel- 
fare, but family net incomes were largely unchanged. As a result, within a two-year follow- 
up period the programs lifted few families above the poverty line. Impact estimates of reduc- 
tions in welfare. Food Stamps, and other benefits generally matched or exceeded impact esti- 
mates of earnings gains. Including estimates of income fi-om the Earned Income Tax Credit 
(EITC) produced little change in this finding for all programs except Portland’s, which attained 
the largest and most consistent gain in total combined income ($238, or $425 including the EITC 
estimate, for year 2 of the follow-up) and also produced a small increase in the proportion with 
combined incomes above the poverty level (4 percentage points, or 7 percentage points including 
the EITC estimate, in year 2). 
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Although no programs had pervasive negative effects on sample members in the 
first two years of follow-up, some individuals were adversely affected. In year 2 of follow-up 
six programs (some employment-focused and some education-focused) produced small increases 
in the proportion of sample members with combined income from welfare, Food Stamps, and 
earnings equivalent to less than 50 percent of poverty levels. In addition, several programs (rep- 
resenting both types of approaches) increased the rate at which individuals left welfare without a 
job (although other persons in their household might have been working). Finally, some pro- 
grams that increased employment also decreased family health insurance coverage (as reported 
by parents) and increased out-of-pocket child care expenditures. 

The programs did not have widespread, large, or consistent two-year effects on the 
children of sample members, but positive and negative effects occurred in some programs. 

No programs in the evaluation provided direct services (with the exception of child care assis- 
tance) to children. Program-produced changes in the lives of sample members (virtually all 
mothers) may, nevertheless, influence the well-being of children. There is evidence that some of 
the programs affected the likelihood of at least one child in a family having behavioral, educa- 
tional, or health and safety problems. There was not, however, a consistent pattern of benefit or 
harm to children. In addition, employment- and education-focused programs did not appear to 
affect children differently; there was no consistent evidence that one particular approach affected 
children more or less or was more likely to help or harm children. 

Several employment- and education-focused programs attained at least moderate 
two-year employment and earnings gains for the “most disadvantaged” sample members. 

Five programs (Portland, Grand Rapids LFA and HCD, and Riverside LFA and HCD) increased 
employment and earnings for individuals who at study entry did not have a high school diploma 
or GED, had not worked in the prior year, and had been on welfare cumulatively for two years or 
more. These five programs and two others (Detroit and Columbus Integrated) also reduced the 
amount of time that the most disadvantaged individuals spent on welfare during the two-year fol- 
low-up period. Notably, for individuals who entered the study without a high school diploma or 
GED, the two-year employment and earnings impacts of the education-focused programs did not 
exceed those of the employment- focused programs.^’ 

High enforcement programs did not produce the largest two-year impacts, but low 
enforcement programs resulted in only small effects. High enforcement programs, notably 
those in Grand Rapids and Columbus, did not necessarily produce the largest impacts. However, 
the two low enforcement programs — Oklahoma City and, in its early stages, Detroit — yielded 
only small impacts. It thus appears that a minimum level of enforcement by program staff is re- 
quired to produce at least moderate earnings and welfare impacts, presumably because this extra 
“push” is needed in order to engage in program activities those who normally would not partici- 
pate on their own initiative. 

While many programs achieved positive two-year effects on employment, earnings, 
and reduced use of welfare, the Portland program stood out as unusually successful. The 

Portland program substantially increased employment and earnings, helped people to get good 



^'For an extension of these results into the third year of follow-up, see Michalopoulos and Schwartz, 2001. 
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jobs, lowered welfare receipt, and achieved these outcomes for a cross section of sample mem- 
bers. The results are probably due to a combination of factors. While its employment message 
was strong, the program offered high-quality education and training services as well as job 
search, enforced a participation mandate, and had strong job development and placement ser- 
vices. In addition, contextual factors may have contributed to the program’s success. In particu- 
lar, it worked with a less disadyantaged welfare caseload (relative to the other studied programs) 
and operated within a good labor market with a relatively high state minimum wage. 

VIII. Contents of This Report 

This report builds and expands on the two-year findings. Chapters 2 and 3 give important 
background information about the NEWWS Evaluation and its participants and their activities. 
Chapter 2 describes the random assignment research design used to test the effectiveness of the 
programs, the definition and characteristics of the various samples included in this report, and the 
types and sources of data used. Chapter 3 describes the five-year effects of the programs on in- 
creasing participation in work-related activities. The chapter also documents whether programs 
increased the percentage of recipients who earned GEDs or other education credentials after ran- 
dom assignment. Importantly, the chapter expands the discussion in this chapter on the extent to 
which both control and program group members received welfare-to-work program services in 
the last two years of the five-year follow-up period. 

Chapters 4, 5, 6, and 7 examine the five-year economic impacts of the programs, for all 
sample members as well as selected subgroups. These longer-term impacts could reflect an ex- 
tension of the effects of services and experiences in the first two years of follow-up, the effect of 
new services and experiences in follow-up years 3 to 5, or a combination of both. Chapter 4 dis- 
cusses the impacts of the programs on sample members’ employment, earnings, job stability, and 
job quality. The chapter investigates whether employment- or education-focused programs fared 
better and what caused increases in average earnings: putting to work welfare recipients who 
would not have found jobs on their own, improving job quality for those who would have been 
employed anyway, or both. Chapter 5 presents impacts on welfare and Food Stamp receipt and 
payments, determining whether the programs achieved welfare savings and whether they did so 
by increasing the speed or frequency of welfare exits or by decreasing average grants for those on 
public assistance. Chapter 6 looks at earnings gains and welfare reductions from the perspective 
of sample members and presents impacts on individuals’ combined income from earnings and 
benefits, level of self-sufficiency, and prospects for longer-term economic security. Chapter 7 
determines the effects of alternative program strategies for different subgroups of welfare recipi- 
ents. It explores the degree to which programs helped groups of the welfare population likely to 
have different capacities to find work on their own: those who had limited education credentials, 
those who were more disadvantaged (without recent work experience and who had been on wel- 
fare for two or more years), and those who were less disadvantaged. 

Chapters 8, 9, and 10 are concerned with several noneconomic family outcomes. Chapter 8 
examines the ways that programs affected sample members’ health care coverage, for adults as well 
as children in the families studied. Chapter 9 discusses the effects of the programs on individuals’ 
household and personal circumstances, examining impacts on marital status, the structure and com- 
position of families and households, fertility, and barriers to work from and abuse by intimate part- 
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ners and others. Chapter 10 looks at the effects of these welfare-to-work programs on sample mem- 
bers’ work-related child care arrangements and on children’s daily activities. 

Chapters 1 1 and 12 examine the effects of the programs on children. Chapter 1 1 presents 
impacts on children of all ages in four of the sites using limited measures of child well-being. 
Chapter 12 presents impacts on a subset of children in three sites who were preschool-age at the 
start of the NEWWS Evaluation and generally between ages 8 and 10 at the five-year follow-up 
point; these analyses use in-depth child well-being measures, constructed from information sup- 
plied by parents, elementary school teachers, and the children. 

Finally, Chapter 13 presents a benefit-cost analysis for each program studied, weighing 
benefits and costs from the perspectives of government and the sample members themselves. 



Chapter 2 



Research Design, Sample Characteristics, 
Data Sources, and Analysis Issues 



The primary aim of the NEWWS Evaluation was to test and compare the effectiveness of 
a v^ety of welfare-to-work approaches in different locales. This chapter describes the research 
designs employed, the samples of people studied and their characteristics, and the types of data 
used in the report. It concludes with guidelines for interpreting the results presented in the chap- 
ters that follow. 

I. Research Design 

To assess the effectiveness of different welfare-to-work strategies, the evaluation used a 
random assignment research design. In each of the seven sites in the evaluation, people who 
were required to participate in a welfare-to-work program were assigned, by chance, either to a 
program group, which had access to employment and training services and whose members were 
required to participate in the program, or to a control group, which received no program services 
and whose members were not subject to a participation requirement but could seek out similar 
services on their own in the community.' Program group members who did not comply with the 
participation mandate risked incurring a sanction, that is, having their welfare grant reduced. 
Control group members, in contrast, could not be sanctioned because of the control embargo that 
precluded them from participating in program activities. Throughout the report, the program and 
control groups are referred to as research groups and the people in them as sample members. The 
random assignment design ensured that there were no systematic differences between the back- 
ground characteristics of program and control group members when they entered the study. Thus, 
any subsequent differences between the groups’ outcomes (called impacts) can be attributed with 
confidence to the effects of the programs. 

Sample members in each research group were tracked over a follow-up period of five 
years after their date of random assignment. Average outcomes for control group members (such 
as employment and welfare receipt) after random assignment represent what could be expected of 
welfare recipients had they never enrolled in a welfare-to-work program. Past studies have shown 
that many people to whom welfare-to-work programs are targeted will leave welfare and find 
work on their own, that is, without being assigned to a welfare-to-work program. 

The differences between outcomes for the program and control groups represent the im- 
pacts or effects of each program. Unless otherwise noted, all “increases” and “decreases” re- 
ported in this document refer to such program-control differences. 

A. Random Assignment Designs 

Four of the sites implemented a three-way random assignment research design in order to 
test the effectiveness of two different program approaches. In the three-way design, each person 
was assigned, by chance, to one of two program groups or to a control group. This design is es- 
pecially powerful because members of both research groups have the same background character- 
istics and face the same labor market conditions and other environmental factors that can affect a 



'As will be discussed in more detail later in this chapter, in some sites control group members became eligible 
for program services before the end of the five-year follow-up period. 
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program’s success in helping people find jobs and advance toward economic security. In the 
three-way sites, the relative effectiveness of the two programs can be assessed by comparing out- 
comes for the program groups with one another directly, that is, without taking the control group 
into account. To assess the programs’ absolute effectiveness, however, it is necessary to compare 
outcomes for each program with the control group’s outcomes.^ 

Three of these four sites (Atlanta, Grand Rapids, and Riverside) operated two programs 
that were designed — for purposes of the evaluation — to magnify the differences between the 
emplo^ent- and education-focused approaches described in Chapter 1: Labor Force Attachment 
(LFA) programs, which emphasize rapid job placement as the best way for welfare recipients to 
develop their work habits and skills, even if the job pays low wages; and Human Capital Devel- 
opment (HCD) programs, which emphasize that welfare recipients have to develop their “human 
capital,’’ that is, their knowledge and basic skills, through education and training in order to have 
a better chance of finding and keeping jobs and advancing toward well-paid and secure employ- 
ment. In each site, the two program models were implemented to maximize the contrast between 
them, thus making the differences between their effects easier to detect. Figure 2.1 illustrates the 
process by which welfare recipients and applicants in Atlanta and Grand Rapids were randomly 
assigned to the research groups. 

The random assignment process differed in Riverside because California’s welfare rules 
mandated that only people “in need of basic education” — that is, people who lacked a high 
school diploma or GED, scored low on a welfare department math or reading literacy test, or re- 
quired instruction in English as a Second Language (ESL) — could be assigned to the HCD 
group. This constraint meant that whereas the HCD group included only people determined to 
need basic education, the LFA group included both such people and people determined not to 
need basic education. To facilitate direct comparisons between the Riverside LFA and HCD 
groups in this report, the results for the subgroup of LFA group members determined to need ba- 
sic education are provided in addition to the results for the full LFA group. ^ A second conse- 



^The following hypothetical example of a side-by-side evaluation of two program approaches illustrates these 
points. Control group members earned a total of $40,000 on average over five years, compared with $40,000 for 
program group 1 and $35, 000 for program group 2. Direct comparisons of earnings for the two program groups 
suggest that the first program was relatively more effective than the second, because its members earned $5,000 
more on average over five years. However, comparisons with the control group show that neither program was effec- 
tive because neither raised average earnings above the control group level. 

^The Riverside design has implications for calculating the LFA program impacts. Whereas the outcomes for san^le 
members in the other six sites are unweighted, in Riverside the outcomes are weighted averages of the outcomes for 
LFA group members found to need or not to need basic education at random assignment. This weighting scheme com- 
pensates for the overrepresentation of those determined not to need basic education in the LFA and control groups. 

Owing to the Riverside program design, impacts cannot be correctly calculated in an unweighted regression 
model (that is, one that includes all the sample members in Riverside and gives all observations equal weight). In- 
stead, the LFA impact is calculated as (Wneed * BLFAnced) (Wnot * BLFAnot)- In this equation, BLPAnced represents the 
inpact for the “in-need” LFA group members and BLPAnot the impact for “not-in-need” LFA group members. W„ccd, 
the weight for the in-need sample, equals the fraction of LFA group members, HCD group members, and control 
group members who were classified by program staff to be in need of basic education at random assignment, and 
Wnob the weight for the not-in-need sample, equals 1 - Wneed- 

The Riverside LFA impacts were generated using a regression model that included all Riverside sample mem- 
bers, whereas the Riverside HCD impacts were estimated using a regression model that included only LFA, HCD, 
and control group members determined to need basic education. 

For many outcome measures, the report presents the range of control group averages across the seven sites. For 
Riverside, the average for the entire control group will be included in the range, and not the separate average for 
control group members in need of basic education that is used to estimate the impacts of the HCD program. 
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Figure 2.1 

Steps Leading to Random Assignment in Atlanta and Grand Rapids 
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quence of this constraint in Riverside is that the only way to make direct comparisons between 
the results of the Riverside HCD program and those of the programs in other sites in the evalua- 
tion is to focus on the subgroups of people in those other programs who lacked a high school di- 
ploma or GED.'* Figure 2.2 illustrates the process by which welfare recipients and applicants in 
Riverside were randomly assigned to the research groups. 

Unlike the goal in the other sites that used a three-way design, the goal in Columbus was to 
test and compare the effectiveness of two different case management models. In the Traditional 
model, one worker handled the welfare department’s employment and training function and another 
worker handled welfare eligibility and payment issues — often called “income maintenance.” Both 
workers maintained relatively large caseloads. In the Integrated model, a single worker handled 
both the employment and training and income maintenance functions. In the Integrated model, the 
worker maintained a smaller caseload than either of the workers in the Traditional model since, on a 
per client basis, the worker was handling jobs “traditionally” done by two workers. 

The remaining three sites in the evaluation (Oklahoma City, Detroit, and Portland) used a 
two-way random assignment design to test the effectiveness of program models already estab- 
lished in those sites. In other words, instead of implementing a program designed expressly for 
research purposes, as in the three-way sites, program administrators in each of the two-way sites 
determined their welfare-to-work program goals and practices and randomly assigned people to a 
group that entered the program or to a control group.^ A summary of the research designs in all 
seven sites is presented in Table 2.1. 

B. Random Assi£nment Periods and Procedures 

In each site, sample members were randomly assigned over a period of approximately 
two years. Random assignment began in June 1991 in Riverside, California, and ended in De- 
cember 1994 in Portland, Oregon (see Table 2.2). Thus, the results presented in this report cover 
the calendar period from June 1991 (the month of the first Riverside sample member’s entry into 
the study) to December 1999 (the last month of the follow-up period for the last sample member 
in Portland to be randomly assigned). Throughout the report, the five years of the evaluation’s 
follow-up period are labeled year 1, year 2, and so on. These labels refer not to calendar years but 
to years after random assignment; for example, year 1 refers to the first year after sample mem- 
bers were randomly assigned, regardless of whether year 1 began in 1991 or 1994 for individual 
sample members. 



“'Nearly one-quarter of the people in the Riverside in-need subgroup actually had a high school diploma or 
GED. These people were determined to be in need of basic education because they scored low on the math or 
reading portion of the appraisal test or were judged by program staff to need English remediation. See also Ham- 
ilton et al., 1997. 

’See Hamilton and Brock, 1994, for a more detailed description of the research designs in the seven sites. 
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Figure 2.2 

Steps Leading to Random Assignment in Riverside 
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Table 2.2 

Overview of Report Sample Sizes, by Site and Research Group 



Site and Program 


Full Impact 
Sample 


Five-Year Client 
Survey Sample 


Two-Year and 
Five-Year Client 
Survey 
Respondents 


Five-Year Child 
Outcomes Study 
Sample 


Five-Year 
Teacher Survey 
Sample 


Atlanta 


Random assignment period 


01/92-06/93 


03/92-06/93 


03/92-06/93 


03/92-06/93 


03/92-06/93 


Labor Force Attachment 


1,441 


519 


491 


289 


184 


Human Capital Development 


1,495 


594 


565 


367 


226 


Control 


1,497 


552 


527 


311 


193 


Full sample 


4,433 


1,665 


1,583 


967 


603 


Grand Rapids 


Random assignment period 


09/91-01/94 


03/92-01/94 


03/92-01/94 


03/92-01/94 


03/92-01/94 


Labor Force Attachment 


1,557 


535 


506 


214 


144 


Human Capital Development 


1,542 


547 


511 


196 


120 


Control 


1,455 


562 


537 


214 


144 


Full sample 


4,554 


1,644 


1,554 


624 


408 


Riverside 


Random assignment period 


06/91-06/93 


09/91-05/93 


09/91-05/93 


09/91-05/93 


09/91-05/93 


Labor Force Attachment 


3,384 


499 


424 


185 


108 


Human Capital Development 


1,596 


376 


323 


208 


131 


Control 


3,342 


720 


648 


348 


222 


Full sample 


8,322 


1,595 


1,395 


741 


461 


Portland 


Random assignment period 
Program 
Control 
Full sample 


02/93-12/94 

3,529 

499 

4,028 


03/93-02/94 

281 

223 

504 


03/93-02/94 

238 

204 

442 






Columbus 


Random assignment period 
Integrated 
Traditional 
Control 
Full sanple 


09/92-07/94 

2,513 

2,570 

2,159 

7,242 










Detroit 


Random assignment period 
Program 
Control 
Full sample 


05/92-06/94 

2,226 

2,233 

4,459 










Oklahoma City 


Random assignment period 
Program 
Control 
Full sample 


09/91-05/93 

4,309 

4,368 

8,677 










Full sample size 


41,715 


5,408 


4,974 


2,332 


1,472 



SOURCE: MDRC-created database. 
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The differences in the random assignment procedures used in different sites affected the 
composition of the site samples and, thus, the comparability of the results for different sites and 
programs.^ In five of the seven sites, welfare applicants and recipients who were required to par- 
ticipate in a welfare-to-work program (because they met certain demographic criteria: for in- 
stance, had no children below the minimum age set by the program) were randomly assigned 
while attending a program orientation at their local employment and training office. In Columbus 
and Oklahoma City, in contrast, people were randomly assigned at their local income mainte- 
nance office before being assigned to an orientation. 

Not everyone assigned to participate in a welfare-to-work program actually attends an orien- 
tation. Reasons for not attending include leaving welfare shortly after being referred to the program, 
having one’s welfare application denied, or simply failing to show up.^ As a result, the people who 
attend a program orientation may not be representative of everyone in their locale who is required to 
participate in a welfare-to-work program. For example, when the waiting list for orientation “slots” 
is long, the people who find jobs and exit welfare before being randomly assigned are likely to be 
more employable, on average, than people who do not. As a result, those who enroll in the program 
are disproportionately likely to be “disadvantaged.” Data on how many people who were required 
to attend an orientation actually did so are available for three sites (Riverside, Grand Rapids, and 
Columbus). About 66 percent in the Riverside, Grand Rapids, and Columbus Traditional programs 
and about 83 percent in the Columbus Integrated program attended an orientation.^ Because out- 
comes in this report are reported as averages for all sample members in a group — including, in 
Columbus, those who did not ultimately attend an orientation — the relative effects of the Colum- 
bus Integrated and Columbus Traditional programs on sample members’ participation and subse- 
quent employment, earnings, and welfare outcomes reflect not only differences in case management 
strategies but in the capacity to enroll people. 

The Oklahoma City results should be interpreted with the following in mind: Oklahoma 
City, unlike the other sites, randomly assigned only welfare applicants (including those whose ap- 
plication for assistance was not yet approved) to the research groups.^ Moreover, about 30 percent 
of sample members in Oklahoma City were denied cash welfare assistance shortly after being ran- 
domly assigned. Therefore, the impacts for the Oklahoma City program are based on a sample 



®For a discussion of enrollment practices in the sites, see Chapter 1. See also Hamilton and Brock, 1994, pp. 51-55. 

^See Hamilton and Brock, 1994, for a discussion of the implications of orientation attendance. A separate ex- 
perimental analysis of the deterrence effects of a participation mandate and reasons for nonattendance was conducted 
in Riverside and Grand Rapids for the NEWWS Evaluation. For this study people who attended a meeting at income 
maintenance to determine their eligibility for welfare benefits were randomly assigned when income maintenance 
workers determined they were subject to the participation mandate. They entered either a “pre-orientation program 
group” and were assigned to attend a program orientation — or a “pre-orientation control group” and were not as- 
signed. Members of the pre-orientation program group who showed up for their orientation during the sample intake 
period for this study were randomly assigned a second time — to either a program or control group. Only those who 
were randomly assigned to a program or control group at program orientation in Riverside and Grand Rapids are 
included in the analyses presented in this report. See Knab et al., 2001, for estimates of the deterrence effects of as- 
signment to a mandatory welfare-to-work program. 

*Brock and Harknett, 1998; Scrivener and Walter, 2001; and Knab et al., 2001. 

^Although Oklahoma City included nonapplicants in its participation mandate, recipients were not included in 
the evaluation because including them would have required significant alterations to existing welfare department 
procedures. 

'“Storto et al., 2000. 





that included a larger proportion than in the other sites of people who never received a welfare 
payment after random assignment for reasons unrelated to the program. In addition, past research 
has shown that welfare-to-work programs have different effects on welfare applicants than on re- 
cipients, most likely because recipients tend to be more disadvantaged than applicants." 

n. Analysis Samples and Sample Characteristics 

A. Analysis Samples 

Table 2.2 shows the dates of random assignment and sizes of the samples used in this re- 
port, by site and research group. All the analysis samples used in the report are listed and de- 
scribed below. 

Full impact sample. The full impact sample includes 41,715 program and control group 
members from all seven sites, for whom five years of administrative records data were collected 
(Figure 2.3, box A 

Five-Year Client Survey sample. Additional data on outcomes for adults and children 
were collected by interviewing sample members around two years after their date of random as- 
signment and, in four of the seven sites, around their five-year anniversary. This report focuses 
on outcomes from the Five-Year Client Survey (detailed findings from the Two-Year Client Sur- 
vey can be found in the two-year report from the NEWWS Evaluation). The Five-Year Client 
Survey sample (Figure 2.3, box B) includes 5,408 members of the full impact sample in Atlanta, 
Grand Rapids, Riverside, and Portland. In each site, survey selection took place during some, but 
not all, months of random assignment. The survey sample was drawn from members of the full 
impact sample who had earlier been selected to be interviewed at two years, whether or not they 
actually responded to the two-year survey (4,974 sample members answered both surveys, and 
434 responded to only the five-year survey; see Table 2.2).'^ Those selected to be interviewed at 
two years were a stratified random sample of the full impact sample members who were ran- 
domly assigned during the months when the survey sample was selected. 

Certain subgroups were intentionally oversampled to produce large enough samples for 
special analyses such as the Child Outcomes Study (COS) and an intensive examination of adult 
education. Results from all programs in this report have been weighted to reflect the overall 



"Friedlander, 1988. 

'^The sample includes only the 499 control group members in Portland who had a full five-year embargo on the 
receipt of program services (more information on the control group embargo is included at the end of this chapter). 
Also, the sample includes only sample members in Atlanta who were randomly assigned between January 1992 and 
June 1993, excluding those randomly assigned after June 1993. 

Approximately 15,000 more people were randomly assigned than are in the full impact sample. Excluded from 
this report s analysis are people randomly assigned before they attended a program orientation as part of the deter- 
rence study, two-parent (AFDC-UP) families, and teen parents in ftiverside (who faced different program recjuire- 
ments than older sample members). 

'“'See Freedman et al., 2000a. 

’’The Two-Year Client Survey was conducted in all seven NEWWS Evaluation sites and included 9,675 re- 
spondents. For more information, see Freedman et al., 2000a. 

'%r the two-year results of the former, see McGroder et al., 2000; for the results of the latter, see Bos et al., 2001 . 
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demographic characteristics of the larger sample. The survey response rates exceed 70 percent for 
all programs and research groups (and 80 percent in Atlanta and Grand Rapids).’^ 

Child Outcomes Study sample. The COS, which is part of the NEWWS Evaluation, in- 
cludes the families of 2,332 sample members who responded to the Five-Year Client Survey in 
Atlanta, Grand Rapids, and Riverside (Figure 2.3, box C).'* Individuals in these three sites who 
were selected for the Two-Year Client Survey and who had at least one child aged 3 to 5 at ran- 
dom assignment were randomly selected to be part of the COS. Each family included a child 
aged 3 to 5 at random assignment who was randomly selected to serve as the focal child, that is, 
the child about whom the most extensive information was collected. Response rates to the COS 
survey in Atlanta and Grand Rapids exceeded 75 percent; response rates in Riverside were be- 
tween 63 and 67 percent for different research groups.'^ 

Teacher survey sample. In a study of children’s school progress, which is also part of the 
evaluation, COS mothers who were interviewed at the five-year follow-up point were asked for 
permission to mail a survey to the focal child’s current elementary school teacher that asked about 
outcomes such as academic performance. (For details, see Section m.) The teacher survey sample 
includes responses fi'om 1,472 teachers of focal children (Figure 2.3, box D).^° Response rates were 
lower for the teacher survey than for the other surveys, ranging fi-om 37 to 57 percent.^' 

B. Sample Characteristics 

Ethnicity. The racial and ethnic makeup of the full impact sample varied fi’om site to site, 
reflecting general differences in the overall ethnic composition of the counties fi-om which the 
samples were drawn. For example, whereas almost all sample members in Atlanta and Detroit 
were Afiican- American, about one-half of sample members in Grand Rapids, Riverside, Colum- 
bus, and Oklahoma City and two-thirds of those in Portland were white. Only Riverside had a 
substantial proportion (one-third) of Hispanic sample members (see Table 2.3). 

Family structure. Almost all the sample members in the evaluation were single parents. The 
“average” sample member was a 30-year-old single mother with two children.^^ She was likely to 
have had a preschool-age child at random assignment and to have had her first child as a teenager. 



‘^For specific response rates by site and research group, see Appendix Table G.l. 

‘ Mothers and focal children in 2,594 families responded to the Five-Year Client Survey. A total of 262 of these 
families were later dropped from the analysis sarr^le. Of these, 203 families had moved out of the survey area by the 
time of the five-year survey and therefore were not administered the special in-person COS survey sections (a phone 
interview was conducted to obtain information for the sections of the survey that were ad mini stered to all five-year 
survey sample members; these sample members remain in the five-year survey sarr^le). Fifty-seven families were 
dropped because the focal child was not the mother’s biological child; one duplicate case was dropped; and one fam- 
ily was dropped because the focal child was deceased at the five-year follow-up point. 

For specific response rates by site and research group, see Appendix Table G.l. 

A total of 1,489 teachers responded to the teacher survey. Seventeen teacher respondents were dropped from the final 
analysis sanple because they taught focal children who were among the 262 respondents dropped from the COS sanple. 

For specific response rates by site and research group, see Appendix Table G.l. 

As shown in Table 2.3, single fathers, or the husbands of disabled spouses, make up from 3 to 11 percent of 
the full intact sample, depending on site. Female pronouns will be used hereafter to describe sample members be- 
cause most of them are women. 
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Married, living with spouse 1.4 3.3 8.1 8.2 2.7 3.8 1.7 

Separated 20.7 18.4 31.4 22.3 15.8 35.7 21.8 

Divorced 16.1 19.3 26.5 18.6 12.3 25.2 28.6 

Widowed [A TO y 08 \2 LO 0.6 

(continued) 



Table 2.3 (continued) 
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Table 2.3 (continued) 

SOURCE: MDRC calculations from information routinely collected by welfare staff. 
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This characterization does not capture the diversity of the families who were subject to 
program participation mandates in these locales. In particular, it does not reflect important site 
differences in who was required to participate in a welfare-to-work program. Just under one-half 
of sample members in Grand Rapids, Detroit, Oklahoma City, and Portland — where mothers 
with a child as young as age 1 were required to participate — entered the program when their 
youngest child was under age 3. The remainder of the sample in these four sites and the full sam- 
ples in the other three sites were evenly divided between mothers with a youngest child aged 3 to 
5 and those with a youngest child aged 6 or over. In Grand Rapids, Detroit, and Oklahoma City, 
unlike in the other sites, teen parents are included in the full impact sample (see Table 2.1). 

Educational attainment. Between 55 and 66 percent of sample members had a high 
school diploma or GED certificate when they entered the program, and some enrollees in all sites 
had some college or post-secondary schooling. On average, however, sample members had com- 
pleted only 1 1 years of school before random assignment. Those sample members who had a 
high school diploma or GED certificate at random assignment are described in this report as 
graduates', those without a high school diploma or GED are described as nongraduates. 

Employment history. Sample members’ employment history varied by site. Less than 
one-half of sample members in all sites except Oklahoma City had worked at some point during 
the year before random assignment: from 2 1 percent (in Detroit) to 46 percent (in Grand Rapids). 
Not surprisingly, sample members in Oklahoma City, all of whom were welfare applicants, were 
far more likely to have worked in the year before entering the program (69 percent had done so). 

In addition to having little recent work experience, less than one-half of the sample mem- 
bers in Columbus and Detroit had worked full time for six months or more for one employer at 
some point before random assignment; two-thirds to three-quarters of sample members in the 
other sites had done so. 

Past welfare receipt. The majority of sample members in all sites except Oklahoma 
City had already received welfare for at least two years cumulatively before random assign- 
ment. Just 24 percent of those in Oklahoma City, compared with 54 to 74 percent in the other 
sites, had received cash assistance for two years or more. Excluding Oklahoma, between 28 
and 50 percent of sample members had received welfare cumulatively for five years or more 
before random assignment. 

“Most disadvantaged” status. The sample members considered to be the most disadvan- 
taged were those who lacked a high school diploma or GED (or, in Riverside, who were deter- 
mined to need basic education), lacked any work history in the year before random assignment, 
and had already received welfare for two years or more cumulatively before entering the pro- 
gram. The proportion of sample members in the most disadvantaged group ranged from 5 percent 
in Oklahoma City to 25 percent in Riverside and Detroit. 

Housing status. The proportion of sample members who at random assignment were 
living in public housing developments or receiving housing subsidies through such programs 
as the Section 8 rental assistance program was highest in Atlanta (56 percent) and lowest in 
Detroit (7 percent). Some have argued that federal housing policies discourage people from 
working because — from the standpoint of residents of public and subsidized housing, who 
pay rent on a sliding scale — earnings increases mean rent increases. In addition, gross income 
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limits on housing assistance eligibility could cause a newly employed person to lose her hous- 
ing subsidy altogether. 

Compared with people in the other sites, a fairly large proportion (14 percent) of people 
in the Oklahoma City sample lived in emergency or temporary housing — that is, lived in a shel- 
ter or were homeless — when they applied for welfare. Less than 3 percent of sample members 
in the other sites were experiencing this type of hardship at random assignment. 

III. Data Sources 

The outcomes and impacts presented in this report are drawn from four primary data 
sources: unemployment insurance, welfare, and Food Stamp administrative records; surveys of 
sample members that were conducted at the two-year and five-year follow-up points (the Two- 
Year Client Survey and the Five-Year Client Survey); a survey of sample members focused on 
outcomes for children (the Child Outcomes Study survey); and a teacher survey. 

Client characteristic data. Standard personal data, such as educational background and wel- 
fare history, were collected by welfare staff during routine interviews at the time of random assign- 
ment and are available for all 41,715 heads of the single-parent families in the full impact sample. 

Private Opinion Survey. Data on attitudes and opinions about welfare-to-work programs 
and employment prospects were collected through the Private Opinion Survey (POS), a brief, 
self-administered survey that was completed at program orientation in four of the sites (Atlanta, 
Grand Rapids, Riverside, and Portland), and are available for 18,461 respondents in these sites. 
These sample members represent ^3 percent of those randomly assigned in the four sites during 
the periods when the POS was being administered. 

Reading and math tests. Reading and math achievement tests were administered in four 
sites (Atlanta, Grand Rapids, Riverside, and Portland) at random assignment. Test scores are 
available for 20,577 sample members. These sample members represent about 93 percent of 
those randomly assigned in the four sites during the period when the tests were administered.^^ 

Field research. MDRC staff observed all 1 1 programs in operation and interviewed en- 
rollees, case managers, service providers, and program administrators in each site. Information 
was collected about a range of issues, such as management philosophy and structure, the degree 
to which the participation mandate was enforced, the nature of interactions between caseworkers 
and program participants, the extent to which the program was able to work with all those man- 
dated to participate in it, the availability of services, and the relationships that program staff had 
established with outside service providers and income maintenance staff in the sites; 

Unemployment insurance, welfare, and Food Stamp administrative records data. 

Most employment, earnings, and public assistance impacts were computed using automated 
county and state unemployment insurance (UI), welfare, and Food Stamp administrative records 
data. Five years of follow-up data from the UI system are available for all members of the full 



those who did not take the tests, about one-third did not speak English; the rest were unable to remain on 
site to be tested, spoke English but were unable to read or write it, or did not take the test for other reasons. 
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impact sample; five years of follow-up data from welfare and Food Stamp administrative records 
are available for all sample members in all sites except Oklahoma. 

UI earnings, which are recorded statewide, provide unbiased measures of program impacts 
on employment and earnings. These data, however, do not include earnings from out of state; from 
jobs not usually covered by the UI system, such as self-employment, federal employment, or infor- 
mal child care (all types of work that may have been “off the books”); or from employers who do 
not report earnings. Some of the earnings missed by the UI system may be captured by earnings and 
employment data collected through the two-year and five-year surveys. 

In all sites except Riverside, welfare and Food Stamp payments were also recorded state- 
wide, and payments are captured for all sample members except those who moved out of state. In 
Riverside (as everywhere in California), welfare and Food Stamp payments were recorded only 
within each county, which means that payments received by sample members who moved out- 
side the county were not included in the analysis. Although this could lead to an underestimate of 
the payments received in the Riverside sample, it should not bias the impact estimates because 
there is no reason to expect the program and control groups to show different patterns of moving 
between counties. 

UI earnings data are collected by calendar quarter: January through March, April through 
June, and so forth. For purposes of the evaluation, these data were reorganized so that the quarter 
during which a sample member was randomly assigned is always designated quarter 1, followed 
by quarter 2, and so forth. These quarters are then grouped into “years.” Quarter 1 is not included 
in year 1 because it includes some income earned before random assignment, especially for sam- 
ple members randomly assigned near the end of a calendar quarter. Thus, year 1 covers quarters 2 
through 5, year 2 covers quarters 6 through 9, and so forth. Welfare and Food Stamp payments 
were recorded monthly but were grouped into quarters and years to align with the earnings data. 

Two-Year Client Survey and Five-Year Client Survey. As noted in a previous section, 
this report includes the results of a survey administered at the five-year follow-up point and some 
results of a survey administered at the two-year follow-up point. Both the two-year and five-year 
surveys provide information about sample members’ participation in training and education ac- 
tivities, attaimhent of education credentials, views of work and welfare, employment history, in- 
come, receipt of noncash benefits such as health coverage, child care use, living situations, and 
children’s well-being. 

Survey responses are the only source of information about many key outcomes, such as 
participation patterns for control group members, work hours and wages, income from other peo- 
ple in the household, and outcomes for children. For some outcomes, such as employment, re- 
spondents provided information that was also recorded from administrative data. It is possible for 
data from these two sources to differ. Because the five-year survey respondents represent a sub- 
sample of the full impact sample that was selected during a shorter period of random assignment 
months, the impact and survey samples may differ with respect to observed characteristics (such 
as educational attainment or prior work history) or with respect to unmeasured characteristics 
(such as assertiveness or learning style) that might have affected their ability to find and retain 
employment. (For more information on survey response bias and the degree to which the survey 
sample and full impact samples differ, see Appendix G.) 
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In some cases, administrative records data may be more accurate than the survey data. 
The client survey depends on people’s ability to recall information about events or jobs that they 
may have held up to five years prior to being interviewed, and failures of memory can give rise to 
discrepancies between the dates of employment or amounts of earnings reported in the survey 
and reflected in administrative records. In addition, some respondents may have been reluctant to 
provide information on employment and income that could be found in administrative records or, 
alternatively, may have exaggerated their earnings and income. In other cases, however, survey 
data may be more accurate, such as when respondents were working off the books or in short- 
term employment. The survey may also have captured earnings that employers failed to report or 
reported inaccurately to the UI system. (For more information on the differences between UI- 
reported and survey-based measures of earnings, see Appendix H.) 

Additional COS survey data. COS respondents provided information on focal chil- 
dren’s academic functioning, social skills, and health and safety. In addition, mothers and the fo- 
cal children themselves completed a Self-Administered Questionnaire (SAQ). Mothers’ SAQ 
included questions about domestic abuse; children’s SAQ included questions about academic 
functioning and social skills. 

Teacher survey. Current teachers of focal children in the COS were asked to assess them 
with respect to their academic standing, academic progress, school engagement, behaviors requir- 
ing disciplinary action, and social skills. The teacher survey complements the data collected from 
mothers and the children themselves. Reports from teachers and mothers sometimes differ. Pos- 
sible explanations include the following: The children behaved differently in the presence of 
mothers and teachers, mothers and teachers perceived the children’s behavior differently, or 
mothers and teachers based their reports on different criteria. 

Cost data. The cost analysis used data drawn from state, county, and local fiscal records, 
supportive service payment records, administrative records, the Two-Year Client Survey, the 
Five-Year Client Survey, and case file participation records. 

Benefit-cost data. The benefit-cost analysis is based on administrative records data (UI- 
reported earnings, welfare, and Food Stamp payments), Two-Year Client Survey data, Five-Year 
Client Survey data, and published data. 

Published data and agency reports. Published data and reports from government agen- 
cies were used to gather additional information about the environments in each of the sites, in- 
cluding unemployment rates, welfare caseloads, and welfare grant levels. 

IV. Analysis Issues 

A. Calculating Impacts 

As discussed above, control group outcomes in this evaluation represent outcomes ex- 
pected in the absence of a welfare-to-work program. Program-control differences show the effect, 
or impact, of each program. In the sites that conducted side-by-side evaluations of alternative 
program approaches, differences between the outcomes for each program group represent the 
relative effects of each program. 
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Although random assignment minimizes the likelihood of the research groups’ differing 
systematically at the outset, there can be small differences in their average characteristics at 
random assignment. To control for these differences, the outcomes for each research group 
were regression-adjusted using ordinary least squares in all the analyses presented in the chap- 
ters that follow. 

In this report, a difference between the program and control groups with respect to a par- 
ticular outcome is considered statistically significant if the result of a statistical test indicates that 
there is less than a 10 percent probability that the difference occurred by chance (that is, when the 
p-value, or level of significance, of the difference is under .10). Impacts are generally reported 
only if they are statistically significant. This rule is intended to keep researchers from inferring an 
impact where none exists.^'* 

Many analysts have noted that the greater the number of analyses conducted (regardless 
of the outcomes or domains studied), the greater the likelihood of chance findings and, thus, one 
needs to take the number of outcomes examined into account. However, some argue that this is 
relevant only when outcomes are not theoretically independent from each other. There are strin- 
gent statistical tests of multiple dependent variables that automatically adjust for the (limited) 
number of theoretically related outcomes (that is, multivariate analysis of variance, or 
MANOVA), as well as post-hoc corrections to p-values that can be applied to results from multi- 
ple individual analyses of “similar” outcomes (for example, the Bonferroni correction). 

For this report, we have not attempted to adjust for the number outcomes such as em- 
ployment and AFDC receipt that are examined because many of these outcomes are so highly 
statistically significant that they would pass the most stringent statistical correction for the fact 
that many outcomes are being measured. By contrast, because we are less certain about 
whether the nontargeted (child and family) outcomes examined in Chapters 9, 11, and 12 are 
theoretically independent from one another (and, thus, whether we may in part be capitalizing 
on chance by examining multiple measures of the same or similar underlying constructs), we 
calculate and report the number of findings we might expect by chance as if they were inde- 
pendent from one another. The proportion of statistically significant impacts across all family 
outcome measures (in Chapter 9), and by selected categories of these measures, or across all 
child outcome measures (in each of Chapters 1 1 and 12) and across all relevant programs was 
calculated. Specifically, given that the experiment-wise Type I error rate was set at .10, any 
one result will emerge as significant 10 percent of the time owing to chance alone. The number 
of chance significant outcomes were calculated and noted in drawing any conclusions about 
the effects of these welfare-to-work programs. 

Some might argue that a more stringent standard is needed, requiring that the number of 
significant impacts within each program must exceed chance levels or that the number of signifi- 
cant impacts within each domain of child development must exceed chance levels. Because there 



^“'However, inferring that there is no impact when an impact really exists is another error of concern. In an effort 
to guard against this type of error, impacts with a probability between 10 and 20 percent of having arisen by chance 
are also occasionally discussed in the report, though these findings are referred to as program-control differences 
rather than impacts. These program-control differences are discussed if they are comparable in magnitude to a statis- 
tically significant impact of another program on the same outcome or if the impact appears to be part of a pattern of 
increases or decreases relative to the control group. 



is a lack of consensus on this issue among statisticians, and given that a goal of the analyses on 
family and child outcomes was to provide a thorough examination of program impacts, we did 
not adhere to a more stringent standard. 

All impact estimates are based on the entire research sample, including program group 
members who did not participate in program activities (it is likely that nearly all “nonparticipants” 
in the program group encountered the program messages and participation mandates, which may 
have affected their decision to look for work or to leave welfare). Because all sample members are 
included in the analyses, the impacts must be interpreted as being the results of the welfare-to-work 
programs as a whole, not only of participation in specific program services. By the same principle, 
calculations of average earnings and welfare payments — which form the basis of many of the im- 
pact estimates — include sample members who were not employed (that is, earned $0) or did not 
receive welfare (that is, received $0 in welfare). To the extent that a program turns noneamers into 
earners or encourages welfare recipients to leave welfare, excluding these $0 values from the pro- 
gram and control group averages would lead to seriously biased underestimates of program impacts. 
For example, previous research has shown that some welfare-to-work programs dramatically in- 
creased the proportion of people who have earnings without affecting the average earnings of those 
who work. These programs led to a relatively large impact on earnings when all sample members 
were included in the calculation. However, omitting people with $0 earnings from the analysis 
would have suggested that these programs had no impact on eamings.^^ 

Some analyses in this report focus on subgroups of the full impact sample. In one such set 
of analyses, presented primarily in Chapter 7, each site sample is broken down by various back- 
ground characteristics (such as previous work history) measured at the time of random assign- 
ment. The impacts found for these subgroups can be confidently attributed to the programs under 
study because they are based on characteristics measured before anyone entered the program and 
because program and control group members are similar in other respects; that is, the only differ- 
ence between the program and control subgroups with a particular background characteristic is 
exposure to the program. In the language of evaluations, such impacts are experimental. How- 
ever, because they are based on smaller samples of people, the impact estimates for subgroups 
are less likely to be statistically significant than those for the full sample. Other analyses in the 
report compare outcomes such as average hourly wages for program and control group members 
who shared characteristics (such as being employed) acquired after random assignment. These 
nonexperimental comparisons should be interpreted with caution because the research groups 
may differ with respect to measured or unmeasured background characteristics . that affect em- 
ployment and, in turn, hourly wages. The report presents findings from such nonexperimental 
comparisons to explore underl)dng trends in the experimental impact estimates. 

Because data were unavailable, the results for sample members in Atlanta who were ran- 
domly assigned during the last six months of the random assignment period are not presented in 
this report. Similarly, because welfare and Food Stamp data were not available for years 4 and 5 
for sample members in Oklahoma City, the year 4, year 5, and cumulative impacts on welfare 
receipt. Food Stamp receipt, and combined income are not reported for Oklahoma City. 



^^See, for example, the discussion of two-year earnings impacts for Riverside LFA in Freedman et al., 2000a, 
pp. 61-63. 
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As discussed in Chapter 1, changes in the labor market and the environments in which the 
programs operated during the follow-up period could have affected program impacts. In particu- 
lar, the economic expansion that began in the mid 1990s created a strong demand for entry-level 
jobs nationwide. However, because program and control group members in each site experienced 
these changes, it is difficult to know ahead of time which group was most affected by them. On 
the one hand, program impacts on employment and earnings may diminish as more control group 
members find employment. On the other hand, during an economic expansion welfare-to-work 
programs may help people advance more quickly to higher-paying or more stable employment, 
resulting in increasing impacts over time. 

In addition, most of the programs in the evaluation became more employment-focused 
over time. As a result, in the last two years of the follow-up period, people assigned to education- 
focused programs who remained on welfare received services and messages similar to those that 
people in the employment-focused, programs were exposed throughout the follow-up period. 
However, people in the education-focused programs were probably little affected by this evolu- 
tion in program approach because — even under the original program model — many of them 
would have received job search assistance if they had not found employment after completing 
education and training activities.^^ 

B. Control Group Treatment Over Time 

Another development that has direct consequences for assessing the impacts of the 
NEWWS programs studied concerns changes in the treatment of control group members over 
time. As discussed in Chapter 1, in five sites some control group members who were still receiv- 
ing welfare after year 3 became eligible for and were required to participate in welfare-to-work 
program services. In other words, in these sites the control embargo was no longer in effect in 
year 4 and/or year 5. (In Riverside and Portland, the control embargo was in force for the entire 
five-year follow-up period for the control group samples analyzed in this report.) This section 
discusses the treatment of controls in years 4 and 5 of the follow-up period in detail, to assess the 
extent to which the early lifting of the control embargo in five sites affected the impact estimates 
in those sites, if it affected them at all. 

Figure 2.4 presents a time line of control group members’ eligibility for welfare-to-work 
program services in each site, by quarter of random assignment. In Atlanta, Columbus,, and Okla- 
homa City, there was a fixed date on which the control group embargo on welfare-to-work program 
services was lifted. In these three sites, then, controls randomly assigned early in the sample intake 
period would have had little opportunity for exposure to the programs under study while those ran- 
domly assigned later in this period would have had more opportunity. In Grand Rapids, as noted 
above, controls randomly assigned in 1 993 retained their embargo on welfare-to-work program ser- 
vices for a full five years; those randomly assigned prior to 1993 would have had their embargo 
lifted when they reached the end of their particular three-year follow-up period. In this site, as in 
Atlanta, Columbus, and Oklahoma City, control group members thus would have had differing 
amounts of time during the five-year follow-up in which to possibly be exposed to program ser- 
vices. In Detroit, the control group embargo on welfare-to-work program services was lifted when 
controls reached the end of their three-year follow-up period. Consequently, in this site, all controls 
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See Chapter 3 for a discussion of education-focused program group members’ participation in job search activities. 
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Figure 2.4 

Time Line of Changes in Control Group Members' Eligibility for 
Welfare-to-Work Program Services over the Five-Year Follow-Up Period, 
by Quarter of Random Assignment and Site 
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Figure 2.4 (continued) 




Key: Random assignment quarter 

Not eligible for welfare-to-work program services 
Eligible for welfare-to-work program services 



NOTE: Control group members shown in Portland are the randomly selected group of 499 for whom a full five- 
year embargo on welfare-to-work program services was in effect. 
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would have had an equal amount of time during the full follow-up period for possible exposure to 
the programs under study.^^ All control group members in Riverside and a randomly selected group 
of control group members in Portland had a five-year embargo on welfare-to-work program ser- 
vices. (For ease of presentation, the randomly selected group of Portland controls rather than all 
control group members in the site is shown in Figure 2.4.) 

The extent to which control group members actually received welfare-to-work program 
services once their embargo ended is not known. However, for several reasons it is likely that 
relatively few control group members did so. First, only control group members still receiving 
welfare, when their embargo on welfare-to-work program services was lifted in follow-up year 4 
or 5 would have been informed of and required to participate in such services. (Once the control 
embargo was lifted, controls would have been considered to be mandatory for participation in a 
welfare-to-work program, as long as they did not meet any of the exemption criteria established 
under the Family Support Act.) The following proportions of all control group members in each 
site had four years of “no treatment” follow-up, that is, years with no months in which they were 
receiving welfare and were eligible for welfare-to-work program services: 86 percent in Atlanta, 
67 percent in Grand Rapids, 91 percent in Columbus, and 37 percent in Detroit. Similarly, the 
following percentages of controls had five years of follow-up with no months in which they were 
receiving welfare and were eligible for welfare-to-work program services: 54 percent in Atlanta, 
66 percent in Grand Rapids, 76 percent in Columbus, and 36 percent in Detroit. Additionally, for 
the samples used in this report, 100 percent of controls in Riverside and Portland had five years 
of follow-up with no months in which they were receiving welfare and were eligible for welfare- 
to-work program services. (Some of the control group members receiving welfare when the em- 
bargo was lifted may no longer have been considered mandatory for welfare-to-work program 
participation, as all would have been as of random assignment, and probably would not have 
been told about the program or a requirement to participate in it because of changes since study 
entry in their personal situations, such as illness or disability, employment of more than 20 hours 
per week, or the birth of child.)^* 

Second, welfare-to-work programs in most sites had a phase-in schedule for assigning 
people to the programs if they were required to participate in them. It is likely that control group 
members who were required to participate in the programs would have waited up to six months 
before being assigned to a program. 

Finally, once assigned to a program, control group members would have needed to attend 
a program orientation, be assigned to a specific program activity, and participate in that activity 
in order to actually receive welfare-to-work program services. Typically, less than one-half of 



^’While the research design in Detroit specified a full three-year embargo on welfare-to-work program services, 
8 percent of all Detroit controls ended up participating in the new Work First program in follow-up year 3. See Far- 
rell, 2000, for details. 

^*For example, as discussed in Chapter 9, between 12 and 23 percent of control group members, depending on 
the site, reported at the five-year follow-up point that a new baby had been added to their household since random 
assignment; the Family Support Act exempted women with children under age 3 from a mandatory welfare-to-work 
program participation requirement (or, at state option, women with children under age 1). 
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those assigned to welfare-to-work programs as mandatory participants end up actually participat- 
mg m welfare-to-work program services. 

It should be kept in mind, however, that any encounters with welfare-to-work program 
staff would have represented control group members’ first exposure to a mandatory welfare-to- 
work program after random assignment. These encounters and possible participation in welfare- 
to-work program activities may have given control group members an unanticipated boost — to 
look for work, to pursue education or training, or to change other behaviors. While this boost 
would have occurred for program group members shortly after random assignment, and their ex- 
posure to program staff and participation in program activities would have continued for several 
years, the possible effects of this boost for control group members toward the end of the follow- 
up period cannot be ignored. 

As discussed more extensively in Chapter 3, available data suggest that the proportion 
of control group members who actually received welfare-to-work program services in follow- 
up, years 4 and 5 is likely to have been low. According to five-year survey data, which are 
available for four of the seven NEWWS sites, less than 6 percent of control group members 
subject to a five-year embargo received any welfare payments during the fifth year after ran- 
dom assignment and reported participating, at some point during the same year, in activities 
usually uniquely provided by welfare-to-work programs (job search workshops or work ex- 
perience).^'^ In comparison, participation rates measured in the same way were only slightly 
higher (between 7 and 12 percent, depending on the site) for control group members with a 
shorter embargo on program services.^' Measured in a slightly different way, as the proportion 
of controls who ever received welfare in a “non-embargoed” month and participated in a job 
club or work experience activity in the same year, at most 15 percent of the Atlanta controls 
and 7 percent of Grand Rapids controls were likely to have received welfare-to-work program 
services during the five-year follow-up period. 



^*For a complete description of this process in welfare-to-work programs, see Hamilton, 1995. 

^^Several situations could account for this 6 percent. While it is unusual for community college or other non- 
welfare programs to offer such activities, control group members may have found these programs on their own and 
voluntarily enrolled in these activities, a practice permitted under the NEWWS research design. NEWWS field re- 
search suggests that, over time, more non-welfare agencies in the evaluation sites began to offer job search assis- 
tance, particularly in Portland. In addition, while site welfare-to-work program staff were very diligent in screening 
for control group members at points of welfare application or program enrollment, some of these control group 
members could represent exceptions, or a few cases where controls “slipped through” the screening process. Field 
research, however, as well as periodic welfare case file reviews, indicated that the screening procedures in almost all 
sites were tight and that outside of Detroit very few control group members slipped through them. 

^'Expressing these numbers as the proportion of control group members who received any welfare in follow-up 
year 5 rather than as a proportion of all control group members, among sites or groups of controls with a full five- 
year embargo on welfare-to-work program services, the proportion participating at some point during year 5 in ac- 
tivities usually uniquely provided by welfare-to-work programs was 10 percent in Grand Rapids, 4 percent in River- 
side, and 24 percent in Portland (but the denominator, or the number of Portland controls receiving any welfare in 
year 5, was small). In contrast, among sites or groups of controls where the embargo on welfare-to-work program 
services was lifted at some point during the last two years of the five-year follow-up period, the proportion of those 
receiving any welfare in year 5 who reported similar participation was 1 8 percent in Grand Rapids and 22 percent in 
Atlanta. While these two sets of percentages are much higher than those mentioned in the text, the difference be- 
tween them again suggests that the level of contamination with welfare-to-work program services was only somewhat 
higher in sites or among groups of controls where the control embargo was lifted during follow-up year 4 or 5 than 
where the embargo was in effect for the full five-year follow-up. 
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While the level of participation of the control groups in welfare-to-work program services 
appears to be low, the behavior of control group members might have been affected by contact 
with program staff and by the messages they received about the advantages of working as op- 
posed to receiving welfare. In Portland, where a random sample of 499 control group members 
had an extension of the welfare-to-work program services embargo, it is possible to directly 
compare the employment and welfare behavior of control group members who had a full five- 
year embargo on welfare-to-work program services with those who had only a three-year em- 
bargo. As discussed more fully in Chapters 4 and 5, there were some differences in the behavior 
of these two groups. In follow-up year 3, employment levels were almost identical for the two 
Portland control groups. In years 4 and 5, employment increased much faster for Portland control 
group members whose embargo ended at the close of year 3 than for those whose embargo lasted 
the full five years. The employment rates of the former group were 6 percentage points higher in 
year 4 and 3 percentage points higher in year 5 than those of the latter group. Similarly, average 
earnings of the former group were $500 higher per year in these two follow-up years. Average 
amounts of welfare payments received by the two groups of controls in follow-up years 4 and 5 
differed by year in direction and by a small amount ($57 less for the longer-term embargo group 
in year 4 and $186 more in year 5). 

The Portland control group comparisons suggest that in this site the lifting of the embargo 
on welfare-to-work program services after year 3 did affect the behavior of some control group 
members. For this reason, the randomly selected 499 control group members, who had a five- 
year embargo on welfare-to-work program services, are used as the control group in all Portland 
analyses throughout the report.^^ 

Unfortunately, a randomly selected alternative control group, similar to the one in the 
Portland site, does not exist in the other five sites where some control group members were 
eligible for welfare-to-work program services earlier than the end of the five-year follow-up 
period. In four of the sites (Atlanta, Grand Rapids, Columbus, and Oklahoma City) certain co- 
horts of control group members, that is, individuals who were randomly assigned in certain 
months, had much longer control service embargoes than did other cohorts. (See Figure 2.4.) 
In these four sites, there is no easy way to determine what effect the lifting of the embargo had 
on the behavior of controls toward the end of the five-year follow-up period, since the baseline 
demographics of sample members often differed between cohorts and the employment and 
welfare behavior observed for the various cohorts was often different even before the control 
embargo was lifted. In the fifth site, Detroit, all cohorts of controls had an equally long wel- 
fare-to-work program service embargo. 



^^Both control and program group members probably would have been affected by publicity about the 1996 welfare 
law and, in the three sites where the count toward a welfare time limit began during the five-year follow-up, by the mes- 
sages conveyed by welfare staff about the urgent need to find a job and leave welfare. In the three sites where the count 
toward a welfare time limit began during the five-year follow-up (Atlanta, Columbus, and Oklahoma City), the count 
began at the same time that the control embargo on welfare-to-work program services was lifted. 

^^Data that would show direct evidence that the differences in employment behavior between the two control 
groups in Portland are due to exposure to welfare-to-work programs (evidence such as differences in measured pro- 
gram participation rates for the two groups) are not available. Using the smaller, five-year embargoed control group 
to calculate impacts, however, provides the “safest” estimates of the true effects of the Portland program. 
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Program impacts on employment and earnings and other outcomes in the last two years of 
follow-up in a few of the five sites probably would have been somewhat larger had some control 
group members not been exposed to welfare-to-work programs. Impacts would likely have been 
affected more in follow-up year 5 than in year 4. However, as discussed in Chapter 4 of this re- 
port, many programs can continue to have effects long after control embargoes are lifted. Owing 
to the program group’s early exposure to the programs, early gains in employment and earnings 
can continue in the later years of follow-up, reflecting a “head start” experienced by program 
group members.^'* This factor, combined with the findings of low year 5 control group welfare 
receipt and low year 5 control group use of program services, strongly suggests that ending the 
control group embargo earlier than the end of the five-year follow-up period did not change the 
impact findings very much. As a result, in this report all control group members in these five 
sites are included in the estimates of program impacts. Where appropriate, however, impact 
analyses for follow-up years 1 to 3 are separated fi:om those for years 4 and 5. 

Notably, the control group situations described above do not affect the assessments in this 
report of the relative merits of the Labor Force Attachment and Human Capital Development ap- 
proaches in welfare-to-work programs. Owing to the research design in the three-way sites, the 
fact that the control group embargo ended after year 3 in several of these sites does not affect the 
estimates of the relative effectiveness of the LFA and HCD approaches over five years. Random 
assignment ensured that the background characteristics of LFA and HCD program group mem- 
bers did not differ systematically at the time of random assignment, which means that the out- 
comes for the two program groups can be compared directly with one another without taking the 
control group into account (Figures 2.1 and 2.2).^^ 



^^Prior studies of the long-term effects of welfare-to-work programs have demonstrated that even after a control 
embargo on welfare-to-work program services is lifted programs can continue to have impacts, though perhaps di- 
minishing ones, stemming from a labor market “boost” received by program group members early in the follow-up 
period. (See, for example, the five-year effects of the 1980s SWIM program in San Diego, presented in Friedlander 
and Hamilton, 1993.) Many of the NEWWS Evaluation programs examined here provide similar examples. 

^^As noted above, direct comparisons between the LFA and HCD programs in Riverside can be made only by 
comparing the HCD group with those members of the LFA group who lacked a high school diploma or basic skills at 
random assignment. 
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Chapter 3 



Impacts on Employment-Related Services and Degree Receipt 



This chapter examines the extent to which the mandatory NEWWS employment- and 
education-focused programs actually increased sample members’ participation in employment- 
related activities. The findings presented below demonstrate that participation differences be- 
tween program and control group members (in all seven sites with five-year survey data) and be- 
tween LFA and HCD program group members (in the three LFA-HCD sites) are substantial. This 
chapter therefore confirms that the NEWWS Evaluation represents a legitimate test of the rela- 
tive effectiveness of different welfare-to-work program approaches. In addition to participation 
outcomes, the chapter also discusses whether education-focused programs increased the percent- 
age of sample members who attained GED certificates or other education credentials after ran- 
dom assignment, a key impact measure for these types of programs. Results are presented for the 
full sample and for subgroups of people who were high school graduates (“graduates”) or high 
school nongraduates (“nongraduates”) as of random assignment. Data on participation, from the 
Five-Year Client Survey, are available for seven of the NEWWS programs; the LFA and HCD 
programs in Atlanta, Grand Rapids, and Riverside, and the Portland program. 

I. Key Findings 

• A majority of control group members in all four sites — from 55 to 75 per- 
cent — participated in an employment-promoting activity over the five-year 
follow-up period. Almost all of this activity was initiated by the control 
group members themselves. The most common activities in which controls 
participated were vocational training programs and post-secondary educa- 
tion. Despite the potential for controls to be subject to mandatory welfare-to- 
work programs at the end of the follow-up period in Atlanta and Grand Rap- 
ids, there is little evidence that much participation in such programs did, in 
fact, occur. 

• Relative to participation levels found for control group members, all seven 
NEWWS programs increased participation in employment-promoting activi- 
ties — including job search, basic education, vocational training, and post- 
secondary education — over the five-year follow-up period. For the most part, 
impacts on participation measured earlier in the follow-up were sustained at 
the five-year follow-up point: In all but two programs cumulative five-year 
impacts were as large as impacts found at the two-year mark. 

• As expected, large impacts on job search participation (of approximately 30 
percentage points) were found in all four employment-focused programs (the 
three LFA programs and the Portland program). All three education-focused 
programs (the three HCD programs) also increased job search participation, 
although generally to a lesser degree. Notably, the Riverside HCD program 
had a large 30 percentage point impact on job search participation, matching 
the impacts found in the employment- focused programs. 
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• All three HCD programs had large impacts on participation in education and 
training activities. For nongraduates, increases were particularly large in basic 
education, and the only other impact on participation found for this subgroup 
was an increase in post-secondary education in Grand Rapids. For graduates in 
the two HCD programs that enrolled them, a large impact on vocational train- 
ing program participation was found in Atlanta, but not in Grand Rapids; both 
programs slightly increased basic education participation for this subgroup. 

• The Portland program, with its employment focus and mix of initial program 
activity assignments, resulted in five-year increases in both job search and 
education participation. While both education subgroups in Portland had large 
impacts on job search participation, nongraduates also experienced a 10 per- 
centage point increase in basic education participation and graduates had a 21 
percentage point increase in post-secondary education. 

• Impacts on the receipt of a high school diploma or GED for nongraduates, fos- 
tered by large increases in basic education participation, were found in all 
three HCD programs as well as in the Portland program. In addition, Portland 
had a notable increase in the proportion of nongraduates who obtained a high 
school diploma or GED and a second education or training credential. None of 
the LFA programs increased diploma receipt for this subgroup. Increases in 
the receipt of any type of education or training credential (generally a trade li- 
cense or certificate) among graduates were found in the Atlanta LFA and HCD 
programs; the Grand Rapids LFA program led to a decrease in such attainment 
for this subgroup. 

II. Analysis Issues 

The analysis of sample members’ levels of participation in potentially employment- 
promoting activities and degree attainment extends the discussion of program dimensions in 
Chapter 1 . Participation levels for program group members and the types of activities in which 
people participate demonstrate how successfully employment- and education-focused programs 
implemented their strategies for self-sufficiency. For the seven programs for which Five-Year 
Client Survey data are available, this chapter will determine how consistently, over the full five- 
year follow-up period, programs increased participation levels or degree receipt beyond what 
welfare recipients would be expected to attain had they never enrolled in a mandatory welfare-to- 
work program. Results for control group members represent the alternative outcomes, and pro- 
gram-control group differences indicate the effect, or impact, of each progr^. It should be 
stressed that a program’s effect on participation depends on the levels attained by members of 
both the program group and the control group. In previous evaluations of welfare-to-work pro- 
grams, firom 20 to 40 percent of control group members enrolled in education and training pro- 
grams on their own over a two-year follow-up period.’ As will be shown, rates of self-initiated 
participation for control group members in this study, where the follow-up is extended to five 
years, is significantly higher. This is important, as programs with similar rates of participation for 



'See, for example, Riccio, Friedlander, and Freedman, 1994, Table 2.4; Kemple, Friedlander, and Fellerath, 
1995, Table 3.5; Hamilton and Friedlander, 1989, Table 3.1. 
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program group members may have very different impacts, depending on how frequently their re- 
spective control group members engaged in employment-related activities on their own. 

During the first two years of follow-up, all 1 1 NEWWS programs generated moderate to 
large impacts on participation,^ and it was expected that these impacts would remain in later 
years as programs continued to enroll in activities individuals who finished their initial assign- 
ment without gaining employment and, in some cases, individuals who left welfare and then re- 
turned. For participants in LFA programs, the intended program path started with job search ac- 
tivities such as job club, intended to last approximately five weeks, followed by short-term edu- 
cation and training only for those unable to find employment. The intended HCD sequence was 
longer-term education and training, generally lasting up to two years, after which clients were 
expected to test their gained skills in the labor market through some type of job search activity.^ 
The Portland program employed a unique, mixed strategy: Program staff assigned most individu- 
als to job search first, while the more disadvantaged members of the caseload were often first re- 
ferred to education and training activities. (See Chapter 1 for a more in-depth discussion of the 
intended assignment patterns in — and the specific activities offered by — each NEWWS pro- 
gram.) As noted in Chapter 1, in all sites the program focus varied somewhat over the five-year 
follow-up period. Most notably among the sites included in the five-year participation analysis, 
the three HCD programs became more employment-focused over time, assigning more individu- 
als to job search and work experience activities. To the extent that this occurred, over time im- 
pacts on job search participation should increase in these programs. In the LFA programs, pro- 
gram group members who did not find employment through job search and work experience 
could have been assigned to education and training, and thus over time education and training 
impacts can be expected to increase in these programs. Furthermore, as the more job-ready left 
welfare for work, programs were left with a more disadvantaged caseload at the end of the fol- 
low-up period. Thus, while impacts were expected to remain over the entire follow-up period, it 
was also expected that the patterns of impacts might change as the types of activities in which 
program group members participated changed over time. 

The end of the embargoes on control group members receiving mandatory welfare-to- 
work program services in Atlanta and Grand Rapids — two of the seven sites examined in this 
chapter is another factor that may have influenced long-term program impacts on participa- 



^See Freedman et al., 2000a, for details. 

An awareness of welfare caseload dynamics is essential in understanding and interpreting welfare-to-work pro- 
gram participation rates. A number of studies have shovm that many welfare recipients cycle on and off the welfare 
rolls, often leaving without any special intervention. For example, some people get jobs on their own or get married. 
To the extent that this occurs among individuals mandated for a welfare-to-work program before they enter their first 
program activity, a site’s overall participation rate will be lowered. This rate will be further lowered to the extent that 
individuals obtain part-time employment, which, if it involves a specified number of hours per week, excuses clients 
firom a program participation requirement. At the same time, welfare-to-work programs may induce some of these 
behavioral changes. For example, a desire to avoid a program participation requirement may lead some individuals 
to find employment or leave welfare sooner than they otherwise would have done, again lowering a site’s participa- 
tion rate if these actions are taken prior to starting an activity. Alternatively, some individuals might feel encouraged 
to remain on welfare longer to take advantage of a program’s opportunities for education and training. Thus, partici- 
pation rates, whether high or low, are influenced by normal welfare caseload turnover as well as by a welfare-to- 
work program’s intervention. In any case, given welfare dynamics, participation rates in these programs should never 
be expected to reach 100 percent. 

See Hamilton et al., 1997, Figure 3.1, Figure 3.2, and pp. 39-44, for a discussion of the sequencing of activities 
in both LFA and HCD programs. 





tion. An analysis of welfare receipt and participation among controls who became eligible for 
welfare-to-work program services shows that most control group members in these four pro- 
grams never received such services during the five-year follow-up period. 

Paiticipation levels presented in the chapter are estimated from survey responses. The 
analysis includes all instances of participation after random assignment, including activities that 
occurred outside the welfare-to-work programs. Most commonly, self-initiated or nonprogram 
participation among program group members occurred after sample members left the welfare or 
program-mandatory rolls; less commonly, they might have participated in a self-initiated activity 
while still enrolled in a welfare-to-work program that their case manager could not approve as a 
program activity because the type or intensity of the activity did not meet the program’s stan- 
dards.^ Sample members are considered to have participated in an employment-related activity if 
they attended for at least one day. Most participants attended for a much longer period.^ The 
Five-Year Client Survey asked about participation since random assignment and in follow-up 
year 5. Thus, cumulative five-year and year 5 participation data are available. 

III. Review of Two-Year Findings 

Two-year participation findings, available for all 1 1 NEWWS programs, provide the basis 
for the longer-term rates, during a time period in which program participation was at its most in- 
tense. Across all 1 1 programs, program group members’ two-year participation rates (in any em- 
ployment-related activity) ranged from 19 to 29 percent in Atlanta, Columbus, and Riverside to 
approximately 40 percent in Grand Rapids, Detroit, Oklahoma City, and Portland.^ Two-year 
participation rates were comparable to participation rates attained in most welfare-to-work pro- 
grams studied in previous evaluations. Program group members most often participated in job 
search and basic education, including Adult Basic Education (ABE), high school completion, and 
General Educational Development (GED) certificate preparation classes, with levels varying by 
program approach and educational attainment subgroup. On their own initiative, without a man- 
date to participate in a welfare-to-work program, control group members participated most often 
in education and training activities (including post-secondary education and vocational training), 
somewhat less frequently in basic education, and least often in job search, work experience, and 
on-the-job training. Control group participation levels were notably high in Detroit, Oklahoma 
City, and Grand Rapids. 

The four employment-focused programs produced large gains in job search participa- 
tion — between 27 percentage points in Grand Rapids LEA and 32 percentage points in Port- 
land — for both graduates and nongraduates. All education-focused programs except for De- 
troit and Oklahoma City achieved large increases in basic education participation for non- 
graduate sample members; however, education-focused programs had little effect on increasing 
participation in employment-related training for graduates. The three HCD programs and Port- 



*In the benefit-cost analysis in Chapter 13, activities outside the programs are called “out-of-program” activities. 

*For detailed descriptions of the length and intensity of participation, including total hours of participation, in all 
1 1 NEWWS programs, see the following: Hamilton et al., 1997, Tables 5.5 and 6.4 (for the LFA and HCD programs 
in Atlanta, Grand Rapids, and Riverside); Scrivener et al., 1998, Table 3.4 (for Portland); Storto et al., 2000, Table 
2.2 (for Oklahoma City); Farrell, 2000, Table 3.5 (for Detroit); and Scrivener and Walter, 2001, Table 3.4 for Co- 
lumbus. 

’See Freedman et al., 2000a, Appendix A.l, for complete two-year participation rates and in^acts. 
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land’s employment-focused, varied first activity program produced moderate to large increases 
in the attainment of a GED certificate among welfare recipients who lacked this credential at 
random assignment. 

In sum, the two-year participation levels indicate that all sites except Oklahoma City and 
Detroit, both of which were low enforcement, successfully implemented their self-sufficiency ap- 
proach. All LFA programs generated large increases in job search, and all HCD programs generated 
large increases in education and training activities. Education-focused program impacts were con- 
centrated among nongraduates, who were most often assigned to basic education. Portland’s mixed 
approach achieved large increases in job search and education and training activities. 

Sanction rates were also examined at the two-year follow-up. In general, there was no 
clear association between a program’s level of sanctioning and participation rates or impacts. 
Among the three programs that had the smallest increases in participation in any activity, two 
were low enforcement, while the third — Grand Rapids LFA — sanctioned a larger percentage 
of sample members than any other program. The two programs with the highest participation 
rates. Riverside LFA and Portland, both had mid-level sanctioning rates. 

For the remainder of the chapter, the focus will narrow to the seven sites with five-year 
survey data. When two- and five-year trends are discussed, the sample is further restricted to 
those sample members for whom both two- and five-year survey data are available. 

IV. Five-Year Control Group Participation Patterns 

In a random assignment research design, control group participation levels reflect self- 
initiated patterns of activity in employment-promoting activities. Participation among controls 
does not represent a compromise to the experiment; rather it shows what program group mem- 
bers would have likely done in the absence of a mandatory welfare-to-work program. In Portland 
and Riverside, control group members were kept from receiving welfare-to-work program ser- 
vices for the entire five-year follow-up period. For these two sites the control group participation 
levels presented in Figure 3.1 (and in the tables throughout the chapter) represent entirely self- 
initiated participation in activities that control group members sought out and attended on their 
own in their communities. In Atlanta and Grand Rapids, some of the control group participation, 
specifically participation in job search and work experience activities toward the end of the five- 
year follow-up period among sample members who were still receiving welfare at that point, did 
in fact occur in the context of a mandatory welfare-to-work program, after embargoes on such 
services were lifted. In Atlanta, it is likely that as much as, but certainly no more than, 15 percent 
of all controls were “exposed” to a mandatory welfare-to-work program in year 4 or 5, while the 
corresponding estimate for Grand Rapids is 7 percent.^ Thus, most controls were kept from re- 



The estimates of the proportion of control group members in Atlanta and Grand Rapids who were “exposed” to 
welfare-to-work program services after embargoes on such services were lifted are considered upper-bound ones for 
a number of reasons, but primarily because of client survey data limitations regarding the ti min g of participation. The 
Five-Year Client Survey did not ask respondents to specify the exact month (as opposed to year) when they partici- 
pated in various activities, so months of post-embargo welfare receipt cannot be lined up directly with participation 
spells. The method of calculating these estimates thus erred on the side of counting as “exposure” some participation 
that, in fact, may have occurred while an individual was no longer receiving welfare. 
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National Evaluation of Welfare-to-Work Strategies 
Figure 3.1 

Two-Year and Five-Year Participation Rates in Employment-Related Activities 

for Control Group Members 




Percentage Points 



Job search 




Percentage Points 




Percentage Points 

SOURCES: MDRC calculations from the Two-Year and Five-Year Client Surveys. 
NOTES: See Appendix A.2. 
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ceiving program services. Control participation in mandatory welfare-to-work program activities 
was further limited in several respects; First, it was limited to the end of the follow-up period (as 
late as the last quarter of follow-up year 5 for some controls in Atlanta); second, it was limited to 
job search or work experience activities, since it is unlikely that controls would have been as- 
signed to activities other than these once they became mandatory to participate. 

Primarily on their own initiative, most control group members in the seven NEWWS pro- 
grams participated in an employment-promoting activity during the five-year follow-up period, as 
shown in Figure 3.1. In all sites, controls mainly participated in education and training activities; 
across all sites from 35 to 55 percent of controls participated in education or training. More spe- 
cifically, controls primarily enrolled themselves in vocational training programs and post- 
secondary education.^ (Participation rates for these activities are presented in Appendix Table 
B.l.) More than 25 percent of control group members in every site except Atlanta took a post- 
secondary education course during the follow-up period. Participation levels were equally high 
for vocational training programs, ranging from 23 percent in Atlanta to 29 percent in Portland. 
For the most part, control group members’ participation in education and training was consistent 
throughout the follow-up period, although in Atlanta participation was more common in years 3 
to 5 than in years 1 and 2. 

While education and training were the most common activities that controls partici- 
pated in throughout the follow-up period, the largest increases in control group participation in 
years 3 to 5 were in job search. Job search was an activity in which very few controls had par- 
ticipated during the first two years of follow-up (two-year participation rates for this activity 
ranged from 4 percent in Atlanta to 7 percent in Portland). By the end of year 5, depending on 
site, from one-fifth to one-third of control group members had participated in job search. In 
Riverside and Portland, the two sites where control group participation levels represent en- 
tirely self-initiated participation, there were striking increases in job search participation in the 
last follow-up years. Such services were most likely provided to control group members by 
community colleges and other community-based organizations in the context of other types of 
education and training activities in which controls were participating, such as vocational train- 
ing programs and courses at community colleges, after they had left welfare. In Portland, for 
example, the local community college system offered much-marketed job search services to 
low-income individuals independent of the welfare department-associated welfare-to-work 
program. In Atlanta and Grand Rapids, as previously noted, there is some evidence that some 
of the increase in job search participation was due to controls participating in mandatory wel- 
fare-to-work programs; however, even upper-bound estimates of likely program-related par- 
ticipation do not account for the bulk of the increase. 

V. Five-Year Impacts on Participation 

The previous section described control group members’ patterns of participation in 
largely self-initiated employment-promoting activities over the five-year follow-up period. As 



’“Post-secondary education” includes courses taken at community colleges or two-year and four-year colleges, 
including — but not limited to — courses taken for credit toward an associate’s or bachelor’s degree. “Vocational 
training includes classes taken for training in a specific job, trade, or occupation, and does not include courses taken 
at community colleges or two-year and four-year colleges. 
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noted, control participation levels represent the extent to which program group members would 
have likely participated in such activities in the absence of a mandatory welfare-to-work pro- 
grarh. Participation impacts, or program-control differences, represent the degree to which pro- 
gram group members in the seven mandatory NEWWS programs examined in this chapter par- 
ticipated in employment-promoting activities above and beyond the rate at which they would 
have participated in the absence of a mandate to participate. 

A. Impacts for the Full Sample 

Impacts for the full sample on three different measures of participation — participation in 
any activity, participation in job search or job club, and participation in any education or training 
activity — are presented in Table 3.1. Appendix Table B.l presents impacts separately on par- 
ticipation in basic education, post-secondary education, vocational training, and work experience. 

Most program group members (from 74 percent in Atlanta HCD to 84 percent in Port- 
land) participated in some type of employment-promoting activity over the five-year follow-up 
period. Overall participation rates (participation in “any activity”) did not vary by program focus; 
for example, while the Portland employment-focused program achieved the highest overall par- 
. ticipation rate, the Riverside HCD program achieved the second highest rate. All seven programs 
increased overall participation relative to control group levels. The magnitude of impacts varied 
somewhat by program: Most programs increased participation by approximately 20 percentage 
points or more, while the Portland program had a relatively small impact of only 9 percentage 
points. The fact that the Portland program did not generate a larger increase in participation is 
likely due to substantial control group participation in the later follow-up years. Control group 
members in this site more actively participated in self-initiated activities, particularly in years 3 
to 5, than controls in other sites. 

As expected, job search was the most common activity among program group members 
in employment-focused programs; well over half of program group members in all four programs 
participated in this activity over the five-year follow-up period. All four of these programs in- 
creased participation in job search by approximately 30 percentage points or more. Work experi- 
ence (including on-the-job training and unpaid work experience programs) was also a somewhat 
common activity in employment-focused programs, particularly the Portland and Atlanta LFA 
programs. All three LFA programs had moderate impacts on participation in this activity, while 
the Portland program did not increase participation relative to the control group. 



'“Five-year participation rates for program and control group members subject to mandatory welfare-to-work 
programs are generally not available, and thus statistics comparable to those presented in this section largely do not 
exist. Five-year participation rates were calculated in an evaluation of San Diego’s Saturated Work Initiative Model 
(SWIM), a program operated in the mid to late 1980s. Among single parents in SWIM (comparable to the NEWWS 
samples analyzed in this report), 66 percent of program group members participated in activities intended to increase 
their employment during a five-year follow-up compared with 42 percent of controls, resulting in an impact of 24 
percentage points. This measure includes participation in welfare-to-work program activities as well as self-initiated 
participation. However, control group members became mandatory for California’s GAIN welfare-to-work program 
after three years if they were still receiving welfare, so SWIM’s estimates of control group participation levels and 
participation impacts are not directly comparable to those in most NEWWS sites. 
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Table 3.1 

Five-Year Impacts on Participation in Employment-Related Activities 



Site and Program 


Sample 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 








Any activity 




Atlanta Labor Force Attachment 


1,071 


74.9 


54.8 


20.1 *** 


36.7 


Atlanta Human Capital Development 


1,146 


74.2 


54.8 


19.3 *** 


35.3 


Grand Rapids Labor Force Attachment 


1,097 


77.7 


63.9 


13.7 *** 


21.5 


Grand Rapids Human Capital Development 


1,109 


81.1 


63.9 


17.1 *** 


26.8 


Riverside Labor Force Attachment 


1,219 


78.2 


57.4 


20.8 *** 


36.2 


Lacked high school diploma or basic skills 


657 


77.0 


55.1 


21.9 *** 


39.7 


Riverside Human Capital Development 


778 


82.0 


55.1 


26.8 *** 


48.6 


Portland 


504 


83.8 


75.0 


8.8 ** 


11.8 






Job search or job club 




Atlanta Labor Force Attachment 


1,071 


59.2 


30.1 


29.2 *** 


97.0 


Atlanta Human Capital Development 


1,146 


43.7 


30.1 


13.6 *** 


45.4 


Grand Rapids Labor Force Attachment 


1,097 


51.6 


21.3 


30.3 *** 


142.5 


Grand Rapids Human Capital Development 


1,109 


39.5 


21.3 


18.2 *** 


85.6 


Riverside Labor Force Attachment 


1,219 


53.5 


20.4 


33.2 *** 


162.9 


Lacked high school diploma or basic skills 


657 


58.2 


19.6 


38.5 *** 


196.5 


Riverside Human Capital Development 


778 


49.6 


19.6 


30.0 *** 


152.9 


Portland 


504 


65.2 


35.4 


29.8 *** 


. 84.1 






Anv education or training 




Atlanta Labor Force Attachment 


1,071 


42.2 


35.3 


6.9 ** 


19.4 


Atlanta Human Capital Development 


1,146 


61.2 


35.3 


25.8 *** 


73.1 


Grand Rapids Labor Force Attachment 


1,097 


54.4 


55.2 


-0.7 


-1.3 


Grand Rapids Human Capital Development 


1,109 


69.3 


55.2 


14.1 *** 


25.6 


Riverside Labor Force Attachment 


1,219 


50.1 


48.9 


1.3 


2.6 


Lacked high school diploma or basic skills 


657 


46.0 


47.1 


-1.0 


-2.2 


Riverside Human Capital Development 


778 


68.9 


47.1 


21.9 *** 


46.4 


Portland 


504 


67.5 


55.0 


12.5 *** 


22.8 



SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A. 2. 
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Notably, job search participation among HCDs increased markedly in the last follow-up 
years. Across the three programs, five-year job search participation rates ranged from 40 to 50 
percent." As previously noted, these increases were anticipated for several reasons. It is natural 
that with the passage of time in a longer follow-up period, participation in all activities would 
increase. More specifically, over time in these programs additional program group members 
completed their initial education-focused activity assignments, and those who did so were then 
expected to test their skills in the labor market through some type of job search program. Finally, 
as discussed in Chapter 1, these programs became somewhat more employment-focused over 
time. As a result of all these factors, five-year impacts on job search participation were larger 
than two-year impacts in all three HCD programs. Despite these increases, cumulative five-year 
impacts on job search in the Atlanta and Grand Rapids HCD programs were still smaller than 
those in their LFA programs — and in employment-focused programs in general, while the Riv- 
erside HCD program had a job search participation impact of 30 percentage points, roughly equal 
to the impacts found in the four employment-focused programs. 

Education and training remained the most common type of activity for program group 
members in the three HCD programs throughout the five-year follow-up period. Participation 
levels in all three programs were in the 60 to 70 percent range on this aggregate measure. Two of 
the three HCD programs, Atlanta and Riverside, had impacts of greater than 20 percentage points 
on overall education and training, while the third. Grand Rapids, had a more modest impact of 14 
percentage points. 

The aggregate “education and training” measure includes participation in basic education, 
post-secondary education, and vocational training. Participation rates and impacts were also ex- 
amined for each of these activities separately. (See Appendix Table B.l.) Notably, all three pro- 
grams increased basic education participation for the full sample. The Riverside HCD program, 
which enrolled only nongraduates or those deemed by program staff to be in need of basic educa- 
tion at the time of study entry, increased basic education participation by 3 1 percentage points. 
The other two HCD programs, Atlanta and Grand Rapids, enrolled both graduates and nongradu- 
ates, and these programs had more modest gains when both subgroups are considered together. 
(Impacts are discussed separately for graduates and nongraduates, below.) 

For the full sample, impacts on vocational training and post-secondary education were not 
as consistent across the HCD programs and, where they did occur, were not as large as impacts 
on participation in basic education. The HCD programs in Atlanta and Grand Rapids increased 
participation in vocational training for the full sample, although the latter impact was just below 
the threshold for statistical significance. No HCD program increased the percentage of sample 
members who took a course at a community, two-year, or four-year college. 

While there was substantial education and training participation among LFAs — roughly 
half of all program group members in the LFA programs participated in an education or training 
activity over the follow-up period — impacts on education and training participation were much 
less common and, where they did occur, smaller in the LFA programs than in the HCD programs. 



"Specifically, the Atlanta HCD program’s five-year job search participation rate was 3.6 times greater than its 
two-year job search participation rate. In the Grand Rapids and Riverside HCD programs, five-year job search par- 
ticipation rates were 2.9 and 2.7 times greater, respectively, than two-year rates. 
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Of the three LFA programs, only Atlanta increased overall participation in education and training 
activities. This impact was due to a 10 percentage point increase in basic education participation, 
as the program did not increase post-secondary education or vocational training participation. 

The Portland program, while employment-focused, differed from the LFA programs in 
that case managers did not always assign individuals to job search as their first activity, and the 
program made heavy use of available short-term education and training programs. Portland staff, 
more so than case managers in the LFA programs, emphasized education and training as a means 
of building skills necessary to acquire higher-paying, stable jobs. This emphasis is reflected in 
program group members’ participation: Over the five-year follow-up period, 42 percent of pro- 
gram group members took a course at a community, two-year, or four-year college, and 25 per- 
cent participated in a vocational training program. While the program did not increase vocational 
training participation relative to control group levels, the program did increase college participa- 
tion by a notable 16 percentage points. 

B. Impacts by Education Subgroup 

Consistent with the findings for the full sample, all programs had substantial impacts on 
participation in any activity for both educational attainment subgroups, of roughly equal magni- 
tude to those found for the full sample. These impacts were statistically significant in all sites 
except for Portland, where the lack of statistical significance is likely a product of small sample 
size. All seven programs also had impacts on job search participation, equal in size to those 
found for the full sample, for both subgroups. (See Tables 3.2 and 3.3 for impacts on participa- 
tion in any activity, job search, and any education or training activity, by educational attainment 
subgroup. Impacts on participation in basic education, post-secondary education, vocational 
training, and work experience are presented in Appendix Tables B.2 and B.3.) In the paragraphs 
that follow, impacts on participation in specific activities are discussed for both high school 
graduates and nongraduates. 

1. High School Nongraduates 

All three HCD programs assigned nongraduate sample members to basic education as 
their first activity, and the majority of nongraduate HCDs (approximately 60 to 70 percent) in all 
three programs participated in basic education over the five-year follow-up period. These partici- 
pation levels resulted in impacts ranging from 27 percentage points in Grand Rapids to 42 per- 
centage points in Atlanta. A small increase in post-secondary education participation in Grand 
Rapids was the only impact found on participation in any education or training activity beyond 
basic education for this subgroup. (See Appendix Table B.2.) 

Atlanta LFA and Portland, the only two employment-focused programs to increase educa- 
tion and training participation for the full sample, also increased this tj/pe of participation for 
nongraduates. The Atlanta LFA program had a 16 percentage point increase in basic education 



For both subgroups, the impacts on participation in any activity are just above the threshold for statistical signifi- 
cance. The sample sizes for these subgroups, including program and control group members, are small: The nongraduate 
subgroup consists of only 163 sample members, while the graduate subgroup consists of 334 sanple members. 
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Table 3.2 



Five-Year Impacts on Participation in Employment-Related Activities 
for Sample Members Without a High School Diploma or GED at Random Assignment 



Site and Program 


Sample Program Control Difference 

Size Group (%) Group (%) (Impact) 


Percentage 
Change (%) 








Anv activity 




Atlanta Labor Force Attachment 


407 


70.1 


49.8 


20.3 *** 


40.7 


Atlanta Human Capital Development 


437 


74.3 


49.8 


24.5 *** 


49.2 


Grand Rapids Labor Force Attachment 


434 


77.0 


64.1 


13.0 *** 


20.2 


Grand Rapids Human Capital Development 


468 


85.0 


64.1 


20.9 *** 


32.6 


Riverside Labor Force Attachment 


657 


77.0 


55.1 


21.9 *** 


39.7 


Riverside Human Capital Development 


778 


82.0 


55.1 


26.8 *** 


48.6 


Portland 


163 


82.0 


71.4 


10.7 


14.9 






Job search or job club 




Atlanta Labor Force Attachment 


407 


56.6 


27.7 


28.9 *** 


104.4 


Atlanta Human Capital Development 


437 


43.2 


27.7 


15.5 *** 


56.1 


Grand Rapids Labor Force Attachment 


434 


51.1 


22.6 


28.5 *** 


126.4 


Grand Rapids Human Capital Development 


468 


39.3 


22.6 


16.7 *** 


74.0 


Riverside Labor Force Attachment 


657 


58.2 


19.6 


38.5 *** 


196.5 


Riverside Human Capital Development 


778 


49.6 


19.6 


30.0 *** 


152.9 


Portland 


163 


64.8 


36.4 


28.4 *** 


78.1 






Any education or training 




Atlanta Labor Force Attachment 


407 


40.3 


31.0 


9.3 ♦♦ 


30.0 


Atlanta Human Capital Development 


437 


65.4 


31.0 


34.4 ♦♦♦ 


111.3 


Grand Rapids Labor Force Attachment 


434 


57.4 


54.4 


3.0 


5.5 


Grand Rapids Human Capital Development 


468 


75.9 


54.4 


21.5 ♦♦♦ 


39.5 


Riverside Labor Force Attachment 


657 


46.0 


47.1 


-1.0 


-2.2 


Riverside Human Capital Development 


778 


68.9 


47.1 


21.9 ♦♦♦ 


46.4 


Portland 


163 


68.3 


58.0 


10.3 


. 17.8 




SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A. 2. 
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Table 3.3 

Five-Year Impacts on Participation in Employment-Related Activities 
for Sample Members With a High School Diploma or GED at Random Assignment 



Site and Program 


Sanple 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 








Anv activity 






Atlanta Labor Force Attachment 


664 


77.9 


58.1 




29 7 ♦♦♦ 


34.0 


Atlanta Human Capital Development 


709 


73.8 


58.1 




15.7 ♦♦♦ 


27.0 


Grand Rapids Labor Force Attachment 


663 


78.0 


64.0 




14.0 ♦♦♦ 


21.9 


Grand Rapids Human Capital Development 


641 


79.0 


64.0 




15.0 ♦♦♦ 


23.5 


Riverside Labor Force Attachment 


562 


79.8 


60.4 




29 3 ♦♦♦ 


32.0 


Portland 


334 


84.2 


77.1 




7.1 


9.2 






Job search or fob club 




Atlanta Labor Force Attachment 


664 


60.6 


31.6 




29.0 ♦♦♦ 


91.6 


Atlanta Human Capital Development 


709 


44.1 


31.6 




12.4 *** 


39.3 


Grand Rapids Labor Force Attachment 


663 


51.9 


20.5 




32 4 ♦♦♦ 


153.3 


Grand Rapids Human Capital Development 


641 


39.5 


20.5 




19.0 ♦♦♦ 


92.6 


Riverside Labor Force Attachment 


562 


47.3 


21.2 




26.0 ♦♦♦ 


122.7 


Portland 


334 


64.2 


36.0 




28.2 ♦♦♦ 


78.2 






Anv education or training 




Atlanta Labor Force Attachment 


664 


43.8 


38.1 




5.7 


15.0 


Atlanta Human Capital Development 


709 


58.0 


38.1 




29 9 ♦♦♦ 


52.2 


Grand Rapids Labor Force Attachment 


663 


52.9 


55.9 




-3.0 


-5.4 


Grand Rapids Human Capital Development 


641 


65.7 


55.9 




9 g 4(4(4( 


17.6 


Riverside Labor Force Attachment 


562 


55.6 


51.4 




4.2 


8.3 


Portland 


334 


67.5 


53.2 




14.3 ♦♦ 


26.9 



SOURCE: MDRC calculations from the Five-Year Client Siuvey. 
NOTES: See Appendix A. 2. 



ERIC 



- 72 - 



117 



participation and the Portland program had a more modest 10 percentage point gain. None of the 
employment-focused programs increased participation in vocational training or post-secondary 
education for this subgroup. 

2. High School Graduates 

The HCD programs in Atlanta and Grand Rapids enrolled sample members with a high 
school diploma or GED at the time of study entry, and they often assigned sample members with 
these credentials to vocational training programs. Basic education was also an assigned activity 
for some sample members with a diploma or GED certificate if they had low math or reading 
skills. Although assignments to college were not made in HCD programs, in some instances case 
managers approved post-secondary education courses that program group members had already 
enrolled in on their own as fulfillment of the participation requirement. 

In Atlanta, 44 percent of graduate HCDs participated in a vocational training program 
over the follow-up period, resulting in a large impact (17 percentage points) on the measure for 
this subgroup. Participation in post-secondary education was less common, and the program did 
not increase participation in this activity. In Grand Rapids, participation levels were high in both 
college and vocational training for graduates among both program and control group members, 
resulting in no impacts on either activity. Both HCD programs generated small increases in basic 
education participation for this subgroup. (See Appendix Table B.3.) 

As was found for the full sample and for nongraduates, impacts on education and training 
activities for graduates in the three LEA programs were not widespread. None of the three LEA 
programs had a statistically significant impact on the aggregate education and training participa- 
tion measure. The only participation impact found among these programs for this subgroup was a 
small increase in basic education participation in Atlanta. 

The Portland program increased education and training participation by 14 percentage 
points for graduates, principally the result of a large 21 percentage point increase in college partici- 
pation. More than half of the program group members in Portland took a course for credit at a two- 
year or four-year college, which was the only NEWWS program to increase college participation. 

VI. Impacts on Participation in Year 5 

As noted, programs were not expected to continue to enroll individuals in employment- 
promoting activities at the same rate during follow-up years 3 to 5 as they did during years 1 and 
2. If the NEWWS programs were achieving their goal of increasing sample members’ self- 
sufficiency, program group members should have been leaving welfare for employment. As this 
occurred, participation levels should have decreased over the follow-up period, particularly in job 
search and work experience activities. Furthermore, changes in the welfare reform environment 
discussed in Chapter 1 might have led to a simultaneous increase in participation among control 
group members who were still on welfare in later follow-up years. All these factors would lead to 
small or no program impacts on participation in year 5. 

As shown in Table 3.4, year 5 participation rates in “any activity” were similar for pro- 
gram and control group members in all sites (ranging from 18 to 26 percentage points), resulting 
in no statistically significant participation impacts. In year 5 program and control members most 
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National Evaluation of Welfare-to-Work Strategies 
Table 3.4 



Impacts on Participation in Employment-Related Activities in Year 5 



Site and Program 


Sample Program Confrol Difference 

Size Group (%) Group (%) (Impact) 


Percentage 
Change (%) 








Any activity 




Atlanta Labor Force Attachment 


1,018 


24.6 


22.5 


2.1 


9.2 


Atlanta Human Capital Development 


1,092 


20.9 


22.5 


-1.6 


-7.1 


Grand Rapids Labor Force Attachment 


1,043 


19.0 


18.7 


0.4 


2.0 


Grand Rapids Human Capital Development 


1,048 


21.6 


18.7 


2.9 


15.5 


Riverside Labor Force Attachment 


1,072 


26.0 


22.0 


4.0 


18.2 


Lacked high school diploma or basic skills 


582 


24.4 


20.5 


3.8 


18.6 


Riverside Human Capital Development 


685 


19.6 


20.5 


-0.9 


-4.4 


Portland 


442 


20.3 


21.9 


-1.6 


-7.2 






Job search or iob club 




Atlanta Labor Force Attachment 


1,018 


14.5 


11.0 


3.4 * 


31.1 


Atlanta Human Capital Development 


1,092 


13.4 


11.0 


2.4 


21.3 


Grand Rapids Labor Force Attachment 


1,043 


8.9 


5.6 


3.3 ** 


59.2 


Grand Rapids Human Capital Development 


1,048 


8.8 


5.6 


3.2 * 


58.2 


Riverside Labor Force Attachment 


1,072 


10.0 


3.9 


6.1 *** 


157.7 


Lacked high school diploma or basic skills 


582 


9.4 


4.5 


5.0 ** 


111.7 


Riverside Human Capital Development 


685 


10.6 


4.5 


6.1 *** 


136.8 


Portland 


442 


10.6 


10.8 


-0.2 


-1.9 






Anv education or training 




Atlanta Labor Force Attachment 


1,018 


11.0 


13.4 


-2.3 


-17.4 


Atlanta Human Capital Development 


1,092 


10.8 


13.4 


-2.6 


-19.5 


Grand Rapids Labor Force Attachment 


1,043 


10.6 


15.0 


-4.4 ** 


-29.4 


Grand Rapids Human Capital Development 


1,048 


13.7 


15.0 


-1.3 


-8.8 


Riverside Labor Force Attachment 


1,072 


16.0 


18.8 


-2.7 


-14.5 


Lacked high school diploma or basic skills 


582 


14.5 


16.7 


-2.2 


-13.3 


Riverside Human Capital Development 


685 


10.6 


16.7 


-6.1 ** 


-36.4 


Portland 


442 


10.7 


12.4 


-1.7 


-13.7 



SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A. 2. 
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often participated in job search, post-secondary education, and vocational training. Very few 
sample members participated in basic education. (Results not shown in tables.) 

While none of the seven programs increased overall participation in year 5, five of the 
programs (all except Atlanta HCD and Portland) continued to generate small increases in job 
search participation. None of the seven programs, however, increased participation in education 
or training activities in year 5, although there was a statistically significant decrease in the River- 
side HCD program, probably because some Riverside HCD program group members who would 
have participated in this type of activity on their own were instead assigned to job search activi- 
ties by their case managers. 

Year 5 participation levels were substantially higher for sample members who received at 
least one welfare payment in year 5. In more than half of the programs year 5 participation rates 
(in “any activity”) for year 5 welfare recipients exceeded 30 percent. Participation levels were 
particularly high in job search; in many programs year 5 job search participation rates for recipi- 
ents were about twice as high as those found for the full sample. Year 5 participation levels for 
those still receiving welfare indicate that the programs were still actively working with the on- 
welfare caseload at the end of the five-year follow-up period. 

In Atlanta and Portland, job search participation levels were high for control group mem- 
bers still receiving welfare in year 5 as well, with roughly one-fifth of control group welfare re- 
cipients in both sites participating in job search in year 5. As has been discussed, this level of 
control group participation in job search in Atlanta is likely at least partially the result of some 
control group members receiving welfare-to-work program services in year 5. In Portland, as 
noted, this level of control group job search participation is likely due to the marketing and reach 
of the local community college system, which offers job search services to low-income individu- 
als independent of the welfare department-associated welfare-to-work program. 

Year 5 participation rates did not vary by educational attainment subgroup. However, sig- 
nificant increases or impacts in job search participation were concentrated among nongraduates. 
(Results not shown.) This cluster of impacts is not surprising, given that nongraduates have 
longer stays on welfare and were thus more likely to still be on welfare and subject to a manda- 
tory welfare-to-work program participation requirement at the end of the follow-up period. 

VII. Trends Over Time in Participation Impacts 

Figure 3.2 presents two-year and five-year participation rates for program and control 
group members. For each program, the difference between the side-by-side black and white bars 
represents the program impact for the respective cumulative period. In all seven programs, as in- 
dicated by the stars representing statistical significance, program group participation levels were 
significantly higher than control group levels for both years 1 and 2 and years 1 to 5. Notably, 
two- and five-year cumulative impacts were similar. For the most part, impacts that were large 
after two years remained so after five years. Thus, during years 3 to 5, differences between pro- 
gram and control group participation rates were maintained; although, for the most part, impacts 
did not grow. 
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Ever Participated (%) Ever Participated (%) 



National Evaluation of Welfare-to-Work Strategies 
Figure 3.2 

Two-Year and Five-Year Participation in Employment-Related Activities, 

by Research Group 



Atlanta LFA 



Atlanta HCD 



■ Program □ Control ■ Program □ Control 





Grand Rapids LFA 
■ Program □ Control 

100 1 
90 - 




Years 1 and 2 Years 1 to 5 



Grand Rapids HCD 




(continued) 
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Ever Participated (%) Ever Participated (%) 



Figure 3.2 (continued) 



Riverside LFA 



Riverside HCD 



B Program □ Control 




B Program 

100 -| 

90 - 



? 70- 




. Years 1 and 2 



□ Control 







Years 1 to 5 



Portland 




SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A.2. 
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Five-year cumulative impacts were somewhat smaller than those found after two years in 
the Atlanta HCD and Portland programs. In all three HCD programs, five-year impacts on job 
search participation were larger than impacts found after two years. (Results not shown.) 

VIII. Five-Year Impacts on Degree Receipt 

Degree attainment can also play an important role in sample members’ long-term labor 
market and welfare behavior. Individuals who received a GED or trade certificate during follow-up 
may have delayed entry into the labor market while they were attending school. In later years, how- 
ever, those who attain new education credentials may have a better chance of finding a job or ad- 
vancing to higher-paying and more stable employment. It is important to note that while research on 
the economic gains to welfare recipients of skill building, including degree attainment, has been 
rather inconclusive, a substantial body of research exists that shows the general importance of edu- 
cation credentials in the labor market. The outcomes described in this section thus may play an im- 
portant role in the economic outcomes discussed in later chapters and in the future labor market 
success of program group members beyond the follow-up period covered in this report.*^ 

In this analysis degree attainment was analyzed separately for high school graduates and 
nongraduates. For nongraduates, three measures were examined: receipt of a high school diploma 
or GED certificate, receipt of a trade license or certificate, and receipt of a high school diploma 
or GED certificate and a second education or training credential. For sample members with a 
high school diploma or GED at the time of program entry, two measures were analyzed: receipt 
of any education or training credential (either a trade license or an associate’s, bachelor’s, or 
graduate degree) and receipt of a trade license or certificate. 

A. High School Nongraduates 

In all three HCD programs a majority of high school nongraduate sample members partici- 
pated in basic education, including GED preparation classes, and the degree to which these pro- 
grams increased receipt of a high school diploma or GED certificate relative to control group levels 
is a key indicator of the relative success of the basic education component of these programs. As 
shown in Table 3.5, across all sites from 3 to 17 percent of all control group members in this sub- 
group received a high school diploma or GED during the follow-up period. All three HCD pro- 
grams increased program group members’ receipt of this credential. The largest impact — 1 1 per- 
centage points — occurred in the Riverside HCD program, where nearly 20 percent of nongraduate 
program group members received this credential over the follow-up period. The Atlanta and Grand 
Rapids HCD programs had slightly smaller impacts, 7 and 9 percentage points, respectively. While 
the Atlanta HCD program had a moderate impact on this measure, absolute levels of degree receipt 
were quite low in this program for both program and control group members. 

Absolute levels of high school diploma and GED receipt were substantially lower for pro- 
gram group members in the three LFA programs than in the HCD programs, and none of the LFA 
programs significantly increased any measure of degree receipt for high school nongraduates. 



'^Chapter 4 in Bos et al., 2001, presents a synthesis of findings related to this topic and of research on welfare 
populations. In particular, see Mincer, 1974; Polachek and Siebert, 1993; and Sum, Taggart, and Fogg, 1995. 
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National Evaluation of Weifare-to-Work Strategies 
Table 3.5 



Five-Year Impacts on Education and Training Credentials 
for Sample Members Without a High School Diploma or GED at Random Assignment 



Site and Program 


Sanple Program 
Size Group (%) 


Control Difference 
Group (%) (Impact) 


Percentage 
Change (%) 






Received a high school diploma or GED 


Atlanta Labor Force Attachment 


407 


4.1 


3.0 


1.2 


39.0 


Atlanta' Human Capital Development 


437 


10.0 


3.0 


*7 2 ♦♦♦ 


237.6 


Grand Rapids’ Labor Force Attachment 


434 


15.4 


13.1 


2.3 


17.5 


Grand Rapids Human Capital Development 


468 


22.5 


13.1 


9 4 ♦♦♦ 


71.4 


Riverside Labor Force Attachment 


657 


5.9 


5.9 


0.1 


0.9 


Riverside Human Capital Development 


778 


17.0 


5.9 


22 2 ♦♦♦ 


188.5 


Portland 


163 


25.9 


16.8 


9.1 


54.2 






Received a trade license or certificate 


Atlanta Labor Force Attachment 


407 


5.3 


5.1 


0.2 


4.0 


Atlanta Human Capital Development 


437 


6.0 


5.1 


0.9 


17.1 


Grand Rapids Labor Force Attachment 


434 


7.3 


9.7 


-2.4 


-24.6 


Grand Rapids Human Capital Development 


468 


9.7 


9.7 


0.0 


0.4 


Riverside Labor Force Attachment 


657 


11.7, 


10.7 


1.0 


9.4 


Riverside Human Capital Development 


778 


11.6 


10.7 


0.9 


8.5 


Portland 


163 


16.6 


4.4 


12.2 ** 


277.5 






Received a high school diploma or GED 




and a second education or training credential 


Atlanta Labor Force Attachment 


407 


1.3 


1.1 


0.2 


20.3 


Atlanta Human Capital Development 


437 


3.1 


1.1 


2.0 


184.9 


Grand Rapids Labor Force Attachment 


434 


2.6 


3.6 


-1.0 


-28.0 


Grand Rapids Human Capital Development 


468 


4.2 


3.6 


0.6 


16.5 


Riverside Labor Force Attachment 


657 


1.8 


2.5 


-0.6 


-26.2 


Riverside Human Capital Development 


778 


3.4 


2.5 


1.0 


38.5 


Portland 


163 


8.4 


0.3 


8.1 ♦♦ 


3,171.6 



SOURCE: MDRC calculations from the Five-Year Client Survey. 



NOTES: See Appendix A. 2. 
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The Portland program had the most noteworthy effects on degree attainment for non- 
graduates — 25 percent of nongraduate program group members received a high school diploma 
or GED over the follow-up period, ^d the program produced an increase, though one above the 
standard cutoff used in this report for statistical significance, of 9 percentage points on the meas- 
ure.''* The program also had a large impact on the receipt of a trade license or certificate, and for 
this subgroup it was the only program to have such an effect. This finding is somewhat surprising 
given that the program did not have a statistically significant five-year impact on vocational 
training for nongraduates (see Section V). There are several possible explanations: The program 
may have increased the likelihood that program group members enrolled in training programs 
offering trade licenses or certificates rather than nondegree programs, while not necessarily in- 
creasing participation in training programs overall. The program also may have increased the 
likelihood that individuals completed credential-offering programs. Finally, it is possible that im- 
pacts on participation in training found at the two-year follow-up point drove this impact on trade 
license receipt.'^ The most notable finding on degree receipt in this program, however, is that it 
had a substantial impact on the percentage of high school nongraduate sample members who at- 
tained both a GED certificate or a high school diploma and a second education or training cre- 
dential (such as a trade license or college degree) over the follow-up period. Nonexperimental 
research conducted as part of the NEWWS Evaluation has found that high school nongraduate 
participants in adult education have substantially stronger improvements in longer-term earnings 
and self-sufficiency if they also participate in skills training or college, making the Portland im- 
pact on this measure a particularly promising finding.'^ 

B. High School Graduates 

As shown in Table 3.6, education and training credential receipt was prevalent among 
control group members who already had a high school diploma or GED certificate at study entry, 
although less so in both Atlanta programs. Trade licenses were by far the most common type of 
credential earned for this subgroup; very few individuals obtained college degrees. Atlanta HCD, 
the only program to increase graduate sample members’ vocational training participation, also 
increased trade license receipt for this subgroup. The only other program to have this effect was 
the Atlanta LFA program. In Grand Rapids, where graduate control group members’ self- 
initiated participation in post-secondary education and vocational training was common through- 
out the follow-up period, the LFA program actually decreased credential receipt for graduates, 
likely by diverting to job search sample members who would have participated in education or 
training activities on their own (and thus possibly have obtained an education credential). This 
pattern is also reflected in the participation impacts discussed in Section V. 



'''The 9.1 percentage point increase has a p-value of .22, and the lack of statistical significance may be a product 
of small sample size (see footnote 12). 

'^For a more detailed discussion of the types of training programs offered by the Portland program, and two-year 
participation rates and impacts, see Scrivener et al., 1998, Chapter 3. 

'^Among nongraduate sample members who participated in basic education during an initial two-year follow-up 
period, those who went on to participate in post-secondary education or training had an additional $1,542 in earnings 
in the third year of follow-up compared with those who participated only in basic education and did not go on to par- 
ticipate in post-secondary education or training (Bos et al., 2001, Table 6.4). 
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National Evaluation of WeIfare*to-Work Strategies 
Table 3.6 



Five-Year Impacts on Education and Training Credentials 
for Sample Members With a High School Diploma or GED at Random Assignment 

Sample Program Control Difference Percentage 

Site and Program Size Group (%) Group (%) (Impact) Change (%) 

Received any education or training credential 



Atlanta Labor Force Attachment 664 

Atlanta Human Capital Development 709 

Grand Rapids Labor Force Attachment 663 

Grand Rapids Human Capital Development 64 1 

Riverside Labor Force Attachment 562 

Portland 334 

Atlanta Labor Force Attachment 664 

Atlanta Human Capital Development 709 

Grand Rapids Labor Force Attachment 663 

Grand Rapids Human Capital Development 64 1 

Riverside Labor Force Attachment 562 



20.4 


13.2 


7.2 •• 


54.7 


26.9 


13.2 


13.7 ••• 


104.2 


18.3 


29.8 


-11.5 ••• 


-38.7 


28.7 


29.8 


-1.0 


-3.4 


20.5 


25.4 


-4.9 


-19.2 


28.9 


22.4 


6.5 


28.9 


Received a trade license or certificate 




16.8 


12.0 


4.9 • 


40.8 


23.6 


12.0 


11.6 ••• 


97.3 


15.5 


22.5 


-7.0 •• 


-31.2 


26.1 


22.5 


3.6 


16.1 


18.1 


22.9 


-4.8 


-21.1 



Portland 334 23.6 18.6 5^0 26.8 

SOURCE: MDRC calculations from the Five-Year Client Survey. 

NOTES: See Appendix A.2. 
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IX. Conclusions 



All three LFA programs in NEWWS were job search first programs, for all individuals who 
were enrolled. Compared with what would have happened in the absence of these programs, the 
LFA programs increased participation in job search — to quite a large degree — for both graduates 
and nongraduates. For the most part, these programs did not increase education and training partici- 
pation or degree receipt, although one program — Atlanta LFA — increased basic education par- 
ticipation for both subgroups and increased trade license or certificate receipt for graduates. 

All HCD programs were education-focused, although activity assignments differed de- 
pending on the educational background and skills level of enrollees. Sample members without a 
high school diploma or GED — as well as some sample members who had this credential but 
had low scores on math or reading achievement tests — were most often assigned to basic educa- 
tion, and basic education participation rates for nongraduates were much higher than control 
group rates in all three programs. The two programs that enrolled graduates in HCD programs 
also increased their participation in this activity, although to a lesser degree. The HCD programs 
were more likely than the LFA programs to result in nongraduates obtaining a high school di- 
ploma or GED at some point during the follow-up period. For those with a high school diploma 
or GED, assignments to and participation in vocational training were most common, although 
only one of the two programs enrolling graduates increased participation in this activity relative 
to control group levels. The difference in assignment patterns for graduate and nongraduate sam- 
ple members is likely explained by the fact that a high school diploma or GED, or in some cases 
a certain score on a reading or math achievement test, is required for entry into most vocational 
training programs. For the graduate subgroup, only the Atlanta HCD program resulted in more 
individuals obtaining an education or training credential (usually a trade license or certificate). 
Notably, the Atlanta LFA program had the same effect for this subgroup. 

As is clear from the findings presented in this chapter, the Portland program was unique 
among these seven NEWWS programs in that program staff employed a mixed strategy, using 
both job search and short-term education and training programs as a first activity, to enhance in- 
dividuals’ self-sufficiency. Program staff encouraged short-term education and training as a 
means of enhancing employability — specifically, as a means of obtaining jobs with higher 
wages and benefits — for all sample members, both graduates and nongraduates. As a result, the 
program increased participation in job search for the full sample and for both subgroups, as well 
as increasing participation in education activities across both subgroups. For nongraduates, in- 
creases in education and training participation led to an increase, relative to control group levels, 
in the percentage of sample members who received a high school diploma or GED and a trade 
license or certificate during the follow-up period. 

All seven programs generated impacts on participation of a sufficient magnitude to verify 
that the programs studied provided a good test of the relative effectiveness of different welfare-to- 
work program models, including the side-by-side test of LFA and HCD programs in the three sites 
that simultaneously operated both program types. The subsequent chapters in this report examine 
the economic effects of all 1 1 NEWWS programs on adults, as well as on households and children, 
that were produced by these changes in employment-promoting activities and in degree receipt. 
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Chapter 4 

Impacts on Employment and Earnings 

Welfare-to-work programs have two primary goals: reducing welfare use and increasing 
employment. This chapter explores the success of the programs studied in the NEWWS Evalua- 
tion in achieving the second of these goals by presenting their effects on employment and earn- 
ings over the five years after people were randomly assigned. The chapter also investigates 
whether employment-focused programs had larger initial effects than education-focused pro- 
grams, whether the effects of education-focused programs had surpassed those of employment- 
focused programs by the five-year point, and which approach was more successful for high 
school graduates and nongraduates. Special attention is given to a direct comparison of results for 
the LFA and HCD programs in Atlanta, Grand Rapids, and Riverside, because these findings 
provide the most reliable evidence of the relative benefits of each approach. 

I. Key Findings 

• Over five years, nearly every program increased employment and earnings 
relative to what was achieved by their control groups, but the size of the ef- 
fects varied over time and by program. 

• The employment-focused programs produced effects almost immediately, 
while the education-focused programs did not generally have effects until 
more than a year after random assignment. In the middle of the follow-up pe- 
riod most of the programs increased employment and earnings. Effects of both 
types of programs generally diminished during the last two years and were sta- 
tistically insignificant for most programs at the end of the follow-up period. 

• Portland produced the largest, most consistent effects by far: over the five-year 
period, program group members worked 1 .6 quarters more than control group 
members — a 21 percent increase in employment — and their earnings were 
about $5,000 greater on average. Portland’s success may be due to its unique 
combination of a focus on employment, use of both job search and education, 
and emphasis on finding good jobs. 

• The employment-focused LFA programs in Atlanta, Grand Rapids, and River- 
side also increased employment and earnings above control group levels, but 
less than the program in Portland. Earnings gains (the amount by which earn- 
ings for the program group exceeded earnings for the control group) ranged 
from about $1,500 in Grand Rapids to about $2,500 in Atlanta and Riverside. 
Employment gains ranged from 0.7 quarter in Grand Rapids to 1.1 quarters in 
Riverside. 

• Overall effects were smaller for the education-focused programs than for the 
employment-focused programs. Neither of the two programs with low en- 
forcement of the participation mandate (Detroit and Oklahoma City) signifi- 



cantly raised employment above control group levels. Among the other five 
education-focused programs, employment gains over five years ranged from 
0.3 to 0.8 quarter and earnings gains ranged from about $800 to about $2,000. 

• Side-by-side comparisons of the LFA and HCD programs in Atlanta, Grand 
Rapids, and Riverside indicate that the LFA programs had larger effects than 
the HCD programs in all sites in the year after random assignment. The two 
approaches had similar effects after the first year; however, impacts for HCD 
programs did not surpass the effects of the LFA programs at the end of the fol- 
low-up. As a result, HCD programs led to smaller cumulative employment 
and earnings impacts over five years than LFA programs and appear unlikely 
to close the gap in later years. 

• Differences between the LFA and HCD programs were concentrated among 
high school nongraduates, suggesting that job search is a better method than 
education for increasing employment and earnings of nOngraduates, but not 
for high school graduates. Portland, which used a mix of job search and edu- 
cation, had the largest effects for both groups, however. 

II. Expected Effects 

The programs pursued three different strategies to attain the goals of increased employ- 
ment and earnings: job search first, education and training first, and a mix of job search and edu- 
cation as initial activities. 

Employment-focused, job-search- first programs (Atlanta, Grand Rapids, and Riverside 
LFA are examples of this approach) encourage rapid entry into the labor market and are therefore 
expected to boost employment and earnings quickly. Proponents of this approach expect people 
to work their way up to better jobs by gaining experience and skills on the job. If this occurs, then 
initial employment gains may persist or increase over time, and earnings increases may grow lar- 
ger in later years. On the other hand, effects of job-search-first programs may grow smaller over 
time, as control group members begin finding work on their own or if program group members 
work at low-quality and unstable jobs that they quickly lose. 

Education-focused programs (they include Atlanta, Grand Rapids, and Riverside HCD; 
Columbus Integrated and Traditional; Detroit; and Oklahoma City) aim to increase enrollees’ 
skills and credentials before they seek employment. Employment and earnings gains may be de- 
layed while recipients participate in education and training activities. Education-focused pro- 
grams may even reduce employment and earnings initially if people would work in the absence 
of the programs, but enroll in education and training activities instead. For these reasons, effects 
are likely to be smaller for the education-focused programs than for job-search- first programs in 
the short term. Toward the end of follow-up, however, the effects of education-focused programs 
may catch up to or surpass the effects of job- search-first programs, if program group members 
make up for initial forgone earnings by obtaining higher-quality jobs than they would have under 
the job-search- first programs. Employment and earnings gains may never occur, however, if en- 
rollees drop out of education and training activities or if area employers have little demand for 
the skills and credentials that enrollees obtain. 
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Employment-focused programs that use a mix of initial activities (Portland used this ap- 
proach) try to combine the best features of the job-search- first and education- focused approaches. 
For more job-ready individuals, they use the same approach as other employment-focused pro- 
grams by trying to move people into jobs relatively quickly. Case managers, however, have more 
discretion to assign some people to short-term skill-building activities first. If this strategy is suc- 
cessful, boosts in employment should occur early in the follow-up, as people required to look for 
work find jobs. Initial gains may be smaller than they would be for job-search-first programs, 
however, because some enrollees participate in education or training activities before looking for 
work. These programs could achieve especially large gains in employment and earnings later in 
the follow-up by moving a large portion of the caseload into higher-quality jobs. Specifically, 
more job-ready participants (who are asked to look for work) may be able to advance to better 
jobs, and less job-ready individuals (who are allowed to enroll in education and training activi- 
ties) may be able to use their new skills to find better jobs than they would have found if they had 
looked for work initially. If neither element of the employment-focused, varied first activity ap- 
proach is effective, however, or if activities are targeted at the wrong people, effects on employ- 
ment and earnings may be small and should not increase. 

This discussion assumes that welfare-to-work programs affect employment and earnings 
by providing helpful services. It is also possible, however, that sanctions used to enforce the 
mandates encourage people to go to work in order to avoid participating in program activities. If 
this is true, then highly mandatory programs may have similar effects initially regardless of their 
self-sufficiency approach. If education-focused programs have smaller effects than employment- 
focused programs, it may ifidicate that services are responsible for the program impacts rather 
than enforcement of the mandates. 

III. Impacts on Employment and Earnings 

A. Employment and Earnings Over Five Years 

Table 4. 1 provides the first indication of how much the different programs affected the 
number of people who worked, the number of quarters they worked, and their total earnings over 
the five-year follow-up period for each of the 1 1 programs in the NEWWS Evaluation. In addi- 
tion, results are shown for sample members in need of basic education for the Riverside LFA 
program so they can be compared with results from the Riverside HCD program, which included 
only people in need of basic education.' (All tables and figures in this chapter show results based 
on quarterly reports by employers to state unemployment insurance systems.) 

Table 4.1 shows that most control group members worked at some point in the follow-up 
period, even though most of them were never eligible for program services and none of them 
were eligible for services during the first three years after random assignment. For example, 
about 88 percent of control group members in Grand Rapids and 86 percent in Columbus worked 
at some point. Except for Riverside (66 percent), the Oklahoma City control group was the least 
likely to work: about 79 percent of control group members worked at some point. The fact that so 



*As discussed elsewhere, results for Portland use only the 499 control group, members who were prohibited from 
receiving program services for five years. 
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National Evaluation of Welfare-to-Work Strategies 
Table 4.1 



Impacts on Employment and Earnings in Years 1 to 5 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Percentage 
Change (%) 








Ever employed (%) 




Atlanta Labor Force Attachment 


2,938 


83.0 


80.3 


2.8 ♦♦ 


3.4 


Atlanta Human Capital Development 


2,992 


81.2 


80.3 


1.0 


1.2 


Grand Rapids Labor Force Attachment 


3,012 


89.8 


88.3 


1.5 


1.7 


Grand Rapids Human Capital Development 


2,997 


89.5 


88.3 


1.2 


1.4 


Riverside Labor Force Attachment 


6,726 


74.5 


66.1 


8.4 *** 


12.7 


Lacked high school diploma or basic skills 


3,125 


70.5 


61.1 


9 ^ ♦♦♦ 


15.4 


Riverside Human Capital Development 


3,135 


66.9 


61.1 


5 8 


9.5 


Columbus Integrated 


4,672 


87.9 


86.0 


^ 9 ♦♦ 


2.2 


Columbus Traditional 


4,729 


87.0 


86.0 


1.0 


1.2 


Detroit 


4,459 


85.5 


83.0 


2 5** 


3.0 


Oklahoma City 


8,677 


78.8 


79.2 


-0.5 


-0.6 


Portland 


4,028 


85.8 


81.7 


4.1 ** 


5.0 






Average number of Quarters emoloved 


Atlanta Labor Force Attachment 


2,938 


8.5 


7.8 


0.8 *** 


9.7 


Atlanta Human Capital Development 


2,992 


8.3 


7.8 


0.5 ** 


6.9 


Grand Rapids Labor Force Attachment 


3,012 


9.8 


9.1 


Q *7 ♦♦♦ 


8.0 


Grand Rapids Human Capital Development 


2,997 


9.5 


9.1 


0.4 * 


3.9 


Riverside Labor Force Attachment 


6,726 


6.8 


5.6 


Y ^ *** 


20.3 


Lacked high school diploma or basic skills 


3,125 


6.0 


4.7 


j 3 ♦♦♦ 


26.9 


Riverside Human Capital Development 


3,135 


5.5 


4.7 


0.8 *** 


16.6 


Columbus Integrated 


4,672 


10.2 


9.8 


0.4 ** 


4.4 


Columbus Traditional 


4,729 


10.1 


9.8 


0.3 * 


3.1 


Detroit 


4,459 


8.3 


8.0 


0.2 


3.0 


Oklahoma City 


8,677 


6.6 


6.7 


-0.1 


-1.3 


Portland 


4,028 


9.4 


7.8 




21.1 



(continued) 
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Table 4.1 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Percentage 
Change (%) 






Average total earnings (S) 




Atlanta Labor Force Attachment 


2,938 


19,838 


17,380 


2,459 *** 


14.1 


Atlanta Human Capital Development 


2,992 


19,397 


17,380 


2,017 ** 


11.6 


Grand Rapids Labor Force Attachment 


3,012 


22,322 


20,770 


1,552 * 


7.5 


Grand Rapids Human Capital Development 


2,997 


21,616 


20,770 


846 


4.1 


Riverside Labor Force Attachment 


6,726 


17,438 


14,889 


2,549 *** 


17.1 


Lacked high school diploma or basic skills 


3,125 


13,193 


10,912 


2,281 *** 


20.9 


Riverside Human Capital Development 


3,135 


12,273 


10,912 


1,361 * 


12.5 


Columbus Integrated 


4,672 


27,621 


25,566 


2,055 *** 


8.0 


Columbus Traditional 


4,729 


26,977 


25,566 


1,410 * 


5.5 


Detroit 


4,459 


21,968 


20,508 


1,460 * 


7.1 


Oklahoma City 


8,677 


12,868 


12,752 


115 


0.9 


Portland 


4,028 


26,041 


20,891 


5,150 *** 


24.7 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 



O 

ERIC 



-87- 



132 



many control group members worked left little room for the programs to increase the number of 
people who ever worked during the follow-up period. In fact, only three of the programs in- 
creased the number of people who worked by more than 3 percentage points. Moreover, the two 
programs with the largest effects were both in Riverside, where control group members were the 
least likely to work. 

In contrast to their generally small effects on whether people ever worked, the programs 
generally increased how long people worked and how much they earned (see the middle and 
lower panels of Table 4.1). In 9 of the 1 1 programs, the program group worked more quarters on 
average than the control group; moreover, a slightly different combination of nine programs in- 
creased average earnings above the control group level. 

The Portland program had the largest effect by far on number of quarters employed and 
earnings. Whereas control group members worked 7.8 quarters on average during the 20-quarter 
follow-up period, program group members worked 9.4 quarters on average, for an impact of 
more than 1 .6 quarters, or more than 20 percent. The program also increased earnings by more 
than $5,000 over five years, fi'om just under $21,000 on average for the control group to just over 
$26,000 on average for the program group. The magnitude of effects in Portland should not be 
understated: Among welfare-to-work programs studied using random assignment, only the Riv- 
erside GAIN effects have been as large as the effects of Portland.^ Portland’s success may be due 
to its strong focus on employment, its emphasis on finding good jobs, and its mixed use of ser- 
vices that assigned some people to job search and some to education and training. Since it is the 
only program studied in NEWWS that had an employment focus with a mix of activities, it is 
difficult to say how important a factor that was in its success. However, in previous evaluations 
the most effective welfare-to-work programs have generally used similar mixed approaches to 
determining people’s initial activities.'* Portland’s unusual success may also have been bolstered 
by its strong economy, although other programs that were operated in areas of high employment 
growth and low unemployment led to smaller impacts on employment and earnings (see Table 
1 .2). It should also be noted that case managers in Portland were less likely than program staff in 
other sites to refer people with serious personal or family problems or extremely low motivation 



^Program inpacts on average number of quarters employed can be greatly affected by the total number of quar- 
ters of follow-up during which employment is measured. For this reason, some researchers prefer to convert this 
measure to a standard format by dividing average number of quarters employed by the total number of follow-up 
quarters. The result (usually expressed as a percentage) is called the “average quarterly employment rate.” For ex- 
ample, in Portland, program group members worked for pay for an average of 9.4 of the 20 quarters of follow-up, 
whereas control group members were employed for 7.8 quarters on average. These averages are equivalent to quar- 
terly enployment rates of 47 percent (9.4 / 20 x 100) and 39 percent (7.8 / 20 x 100), respectively. Similarly, in Port- 
land the impact on average number of quarters employed (of 1.6 quarters) is equivalent to an impact of 8 percentage 
points in the average quarterly employment rate (1.6 / 20 x 100; or 47 percent - 39 percent). 

^Riccio, Friedlander, and Freedman, 1994; Friedlander and Burtless, 1995; Bloom and Michalopoulos, 2001. 
The Riverside GAIN program increased employment by 9.9 percentage points and earnings by about $1,000 per year 
(Freedman et al., 1996; Hotz, Imbens, and Klerman, 2000). Two other programs came close to Portland over a three- 
year period: the Canadian Self-Sufficiency Project (Michalopoulos and Berlin, 2001), which increased employment 
by 7.1 percentage points and earnings by about $725 per year, and the Connecticut Jobs First program, which in- 
creased employment by 7.8 percentage points but increased earnings by only about $425 per year (Bloom et al., 
2000b). 

‘'Bloom and Michalopoulos, 2001; Michalopoulos and Schwartz, 2001. 
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levels. These individuals did not attend a program orientation and were not included in the re- 
search sample.^ 

The LFA programs, with their requirement that most participants look for work initially, 
had the next largest impacts as a group. Increases in the average number of quarters of work 
ranged from 0.7 quarter (Grand Rapids) to 1.1 quarters (Riverside). In addition, increases in total 
earnings over five years above the control group average ranged from about $1,500 per LFA 
(Grand Rapids) to about $2,500 (Atlanta and Riverside). 

The education-focused programs led to smaller impacts on cumulative measures of em- 
ployment and earnings. Columbus Integrated and Atlanta HCD were the most effective at in- 
creasing total earnings above control group levels. These two programs led to five-year earnings 
impacts of about $2,000 per program group member, which exceeded the increase for only one of 
the four employment-focused programs: Grand Rapids LFA. Every other education-focused pro- 
gram led to earnings gains of less than $1,500 over five years (or $300 per year) — ^ a historically 
small increase. Moreover, the program-control group difference in total earnings was not statisti- 
cally significant in Grand Rapids and Oklahoma City. 

As shown in Table 4.1, the Oklahoma City program was the least effective among the 1 1 
programs in the NEWWS Evaluation, probably reflecting the de facto voluntary nature of the 
program. During much of the follow-up period, the Oklahoma City program was hampered by 
large caseloads and limited resources. The program placed a priority on working with enrollees 
who expressed the greatest interest in participating. In addition, its integrated case management 
progr^ staff spent most of their time performing financial functions (such as helping enrollees 
to find child care) and devoted little time to monitoring participation in employment-related ac- 
tivities. Detroit, which faced similar problems, also ran a de facto voluntary program for much of 
the period. Detroit did not significantly raise employment, although it did increase earnings of 
program group members by more than $1,400 over five years relative to the control group. ^ 

B. Employment by Year 

As discussed earlier, employment-focused programs were expected to boost employment 
and earnings immediately, and effects of education-focused programs were expected to occur 
more slowly, after enrollees completed their education and training activities. Table 4.1 shows 
that the cumulative impacts of education-focused programs fell short of the impacts of most em- 
ployment-focused programs. It is possible, however, that the skills obtained through education 
and training helped program members find such good jobs that the impacts of the education- 
focused programs were continuing to grow at the end of the five-year follow-up period relative to 
the impacts of the employment-focused programs. If this pattern did occur, education-focused 



’This finding is based on field research. On the other hand, results in Chapter 7 demonstrate that Portland’s ap- 
proach led to large earnings impacts for sample members who faced at least one serious barrier to employment. 

^Unlike Oklahoma City, Detroit operated a traditional case management system, in which separate Income 
Maintenance staff deteimined recipients’ eligibility for public assistance and calculated their monthly welfare and 
Food Stamp grants. However, the Detroit welfare-to-work program was not fully staffed during the first years of the 
evaluation, and its case managers reported spending most of their limited time working with clients who were par- 
ticipating on their own initiative and who had requested child care and other supportive services. 
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programs might have larger impacts than employment-focused programs after year 5 and larger 
cumulative gains over a longer follow-up period. Figure 4. 1 and Appendix Table C. 1 show, for 
each of the 1 1 programs for the five years of follow-up, the proportion of program and control 
group members who worked for pay.’ In addition, the figure and table can be used to examine 
separately the period when all control group members were embargoed from receiving control 
group services and the later period. As discussed in Chapter 1 , control group members could not 
receive program services in any of the sites during the first three years after random assignment. 
In Riverside and Portland, control group members in the report sample were embargoed from 
receiving services for all five years. It is important to keep in mind, however, that most evidence 
indicates that ending the embargo is unlikely to have changed the effects of most programs on 
employment and earnings very much (as discussed at the end of this chapter). 

The two Atlanta programs exemplify the basic differences in the effects of employment- 
focused and education-focused programs.* In the LFA program, the attempt to move people into 
jobs quickly produced an immediate payoff. During the first year after random assignment, about 
49 percent of the program group worked compared with 44 percent of the control group, a 5 per- 
centage point difference that was statistically significant. In contrast, the HCD program did not 
significantly increase employment in the first year, as was expected from the fact that many pro- 
gram group members were enrolled in education and training. However, the effects of the HCD 
program increased after year 1, and both the LFA and the HCD programs had similar effects on 
employment in years 2 and 3. 

If education-focused programs are to “make up” the ground lost to employment-focused 
programs in the first year, then they must eventually have larger effects than the employment- 
focused programs. Figure 4. 1 shows that, in fact, the Atlanta LFA program continued to signifi- 
cantly increase employment in year 4, but the HCD program did not. In year 5, neither program 
significantly affected employment. This finding suggests that it is unlikely that the HCD program 
will eventually catch up to or surpass the LFA program with longer follow-up. 

The other employment-focused programs led to even larger impacts on percentage em- 
ployed in year 1. Riverside LFA, arguably the most strongly employment-focused program in the 
evaluation, produced the largest initial increase in employment. (See Appendix Table C.l.) In 
Riverside, about 51 percent of LFAs worked for pay some time during year 1 compared with 34 
percent of the control group. Impacts of the Grand Rapids LFA and Portland programs averaged 
about 10 percentage points above the control group level in year 1. 

As discussed above, the positive effects of employment-focused programs may persist or 
grow larger over time. This pattern did not occur for the LFA programs in Grand Rapids and 



^Results over time are also shown in Appendix Tables C.2 (impacts on total earnings in each year of follow-up), 
C.3 (impacts on cumulative measures of employment and earnings in years 1 to 3), and C.4 (impacts on employment 
and earnings in year 5). Appendix Table C.5 shows employment and full-time employment at the end of year 5 ac- 
cording to responses to the Five-Year Client Survey. Results in those tables confirm that few programs continued to 
have significant effects in year 5. 

®The two Atlanta programs are discussed first because they appear side by side in Figure 4.1 and because they 
used different self-sufficiency approaches. Section IV of this chapter presents a statistical comparison of employment 
and earnings in the LFA and HCD programs in Atlanta, Grand Rapids, and Riverside. 
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Employed (%) Employed (%) 



National Evaluation of Welfare-to-Work Strategies 
Figure 4.1 

Impacts on Employment in Years 1 to 5 

Atlanta LEA Atlanta HCD 

■ Program □ Control ■ Program □ Control 





Grand Rapids LFA Grand Rapids HCD 
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Figure 4.1 (continued) 



Riverside LFA 
I Program □ Control 



Riverside HCD 
Program □ Control 




Year 1 Year 2 Year 3 Year 4 Year 5 



Columbus Integrated 



I Program 



□ Control 



Year 1 Year 2 Year 3 Year 4 Year 5 

Columbus Traditional 

■ Program □ Control 




Year 1 Year 2 Year 3 Year 4 Year 5 



Year 1 Year 2 Year 3 Year 4 Year 5 
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Figure 4.1 (continued) 



Detroit 



■ Program □ Control 




Year 1 Year 2 Year 3 Year 4 Year 5 



Portland 

■ Program □ Control 

80 1 




Year 1 Year 2 Year 3 Year 4 Year 5 
SOURCE: MDRC calculations from state and county £ 
NOTES: See Appendix A. 1 . 
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Riverside, however. For both programs, impacts on employment declined after year 1, although 
for different reasons. In Grand Rapids, LFA employment levels increased between years 1 and 2 
and again between years 2 and 3, but control group employment levels increased even faster. 
This result suggests that the Grand Rapids LFA program succeeded initially by encouraging peo- 
ple who would have worked eventually to take jobs faster. In contrast. Riverside LFA employ- 
ment levels diminished sharply between years 1 and 2, suggesting that a number of LFAs were 
encouraged to find work by the program but left employment quickly and had trouble finding an- 
other job. These differences are also consistent with the fact that Riverside had a fairly weak 
economy during the early part of the follow-up period. 

Most other education-focused programs had small effects initially. Only two of the seven 
education-focused programs — Grand Rapids HCD and Riverside HCD — led to significant im- 
pacts on employment in year 1. For all of the education- focused programs except Oklahoma City 
(where impacts were statistically insignificant in each year), impacts were somewhat larger in 
year 2 or 3 than in year 1 . Disappointingly, the effects of most education- focused programs de- 
clined after year 3. For all education-focused programs except Riverside HCD, employment lev- 
els were not significantly higher for program group members than for control group members in 
year 5. (Section V of this chapter presents further evidence that the end of the control group em- 
bargo is not responsible for declining impacts in most programs in years 4 and 5.) 

Results in Grand Rapids and Riverside confirm that, as in Atlanta, it is unlikely that edu- 
cation-focused programs would surpass employment- focused programs with longer follow-up. In 
Grand Rapids, neither the LFA or HCD program increased employment levels above the control 
group levels in years 4 and 5, while in Riverside, the LFAs in need of basic education had 
slightly larger employment impacts than the HCDs. 

The Portland program led to the most successful year-by-year pattern of employment im- 
pacts: It immediately increased employment (like the LFA programs) but continued to increase em- 
ployment after year 1 (like the HCD programs). In fact, the year 3 impact on percentage employed 
(of 12 percentage points) was more than twice as large in Portland as any other program. Like both 
types of programs, its effects declined after year 3; however, the Portland program again led to rela- 
tively large effects (of 8 percentage points) and was one of only three programs to still have a posi- 
tive effect on employment in year 5. This pattern suggests that Portland combined the best features 
of employment-focused and education-focused programs, assigning job-ready individuals to job 
search but allowing others to get short-term education or training initially. 

Nonetheless, the decline in employment impacts for all programs after year 3 under- 
scores the limitations of both employment- and education-focused welfare-to-work strategies 
that rely primarily on pre-employment services and mandates. More recently, policymakers and 
program administrators have attempted to strengthen the effects of welfare-to-work programs 
by adding a variety of post-employment services and financial incentives aimed at helping 
people stay employed and advance to better jobs. It remains to be seen whether such strategies 
accomplish these goals. 
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C. Earnings by Year 



Figure 4.2 and Appendix Table C.2 show the average annual earnings of program and 
control group members during years 1 to 5. These earnings comparisons provide the most com- 
prehensive measure of the effects of the programs because they reflect differences in the number 
of people who went to work, how long they worked, and their average wages. 

The trends in program effects on earnings are similar to the effects on employment. Em- 
ployment-focused programs had larger initial effects than education-focused programs, but most 
programs of both types significantly raised earnings in years 2 and 3 (the exception was Okla- 
homa City). Earnings impacts for both types of programs declined after year 3 and, especially, 
after year 4. By the end of the follow-up period, only the Riverside LFA and Portland programs 
continued to significantly raise earnings above the control group average. 

The pattern of earnings suggests once again that employment-focused programs will con- 
tinue to produce larger cumulative impacts than education-focused programs. Figure 4.2 and Ap- 
pendix Table C.2 indicate that the LFA and HCD programs had similar effects on earnings in 
year 5, and, if anything, earnings impacts were slightly higher for the LFAs. 

Results for Atlanta LFA and Portland deserve special attention. Like the other employ- 
ment-focused programs (Grand Rapids LFA and Riverside LFA), Atlanta LFA and Portland sig- 
nificantly raised earnings in year 1 . Unlike the other two programs, however, effects on earnings 
were greater in years 2 and 3 than in year 1 . In other words, the pattern of their effects on earn- 
ings over time looked more like the education-focused programs and probably reflects these pro- 
grams’ greater emphasis on short-term education and training than the employment-focused pro- 
grams in Grand Rapids and Riverside. 

The findings for Portland are especially noteworthy. Portlaiid continued to produce un- 
usually large earnings impacts (of $1,200 and $900) in years 4 and 5 — well above the effects of 
any other program. As discussed earlier, the Portland program allowed some people considered 
in need of education to enroll initially in short-term education and training and encouraged job 
search participants to seek out relatively well-paying jobs with an opportunity for advancement. 

D. Employment Stability and Earnings Growth 

As discussed above, in theory both employment- and education-focused programs can lead 
to impacts on measures of employment stability and earnings growth. Participants in employment- 
focused prograins are expected to start working quickly and acquire valuable skills and experience 
on the job that lead to employment stability and advancement. In contrast, education-focused pro- 
grams aim to increase job-related skills and education credentials before employment, thereby help- 
ing people to obtain stable and high-paying jobs. Table 4.2 shows the effects of the 1 1 programs on 
the proportion of sample members who worked in all four quarters of year 5, and the proportion 
who earned $10,000 or more in year 5. Appendix Table C.6 shows the effects of the programs on 
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National Evaluation of Welfare-to-Work Strategies 
Figure 4.2 

Impacts on Earnings in Years 1 to 5 
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Figure 4.2 (continued) 
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Figure 4.2 (continued) 



Detroit 

I Program □ Control 



Oklahoma City 
Program □ Control 




Year 1 Year 2 Year 3 Year 4 Year 5 
Portland 

■ Program □ Control 



Year 1 Year 2 Year 3 Year 4 Year 5 




Year 1 Year 2 Year 3 Year 4 Year 5 
SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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National Evaluation of Welfare-to-Work Strategies 
Table 4.2 

Impacts on Employment Stability and Earning $10,000 or More in Year 5 

Sample Program Control Difference Percentage 

Site and Program Size Group (%) Group (%) (Impact) Change (%) 



Atlanta Labor Force Attachment 2,938 

Atlanta Human Capital Development 2,992 

Grand Rapids Labor Force Attachment 3,012 

Grand Rapids Human Capital Development 2,997 

Riverside Labor Force Attachment 6,726 

Lacked high school diploma or basic skills 3,125 

Riverside Human Capital Development 3,135 

Columbus Integrated 4,672 

Columbus Traditional 4,729 

Detroit 4,459 

Oklahoma City 8,677 

Portland 4,028 

Atlanta Labor Force Attachment 2,938 

Atlanta Human Capital Development 2,992 

Grand Rapids Labor Force Attachment 3,012 

Grand Rapids Human Capital Development 2,997 

Riverside Labor Force Attachment 6,726 

Lacked high school diploma or basic skills 3,125 

Riverside Human Capital Development 3,135 

Columbus Integrated 4,672 

Columbus Traditional 4,729 

Detroit 4,459 

Oklahoma City 8,677 

Portland 4,028 



Emoloved in all four Quarters of year 5 




36.9 


36.6 


0.3 


0.8 


36.5 


36.6 


-0.1 


-0.3 


39.4 


38.7 


0.7 


1.7 


38.3 


, 38.7 


-0.4 


-1.0 


26.3 


23.2 


3 1 *** 


13.5 


23.1 


18.8 


4 3 ♦** 


22.9 


21.6 


18.8 


2.7 * 


14.5 


40.7 


39.4 


1.3 


3.3 


40.4 


39.4 


1.1 


2.7 


37.4 


37.1 


0.3 


0.7 


19.4 


18.7 


0.7 


3.9 


38.2 


33.6 


4.6 ** 


13.6 


Earned $10.,000 or more 




25.7 


25.0 


0.6 


2.6 


24.8 


25.0 


-0.2 


-0.8 


26.0 


25.4 


0.6 


2.3 


25.3 


25.4 


0.0 


-0.1 


18.6 


16.6 


2.0 ** 


12.2 


14.3 


12.0 


2.3 ** 


19.6 


12.7 


12.0 


0.7 


5.9 


34.1 


31.6 


2.5 * 


7.9 


32.2 


31.6 


0.6 


2.1 


28.4 


27.1 


1.4 


5.0 


13.9 


13.2 


0.7 


5.3 


30.4 


26.6 


3.8 * 


14.4 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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two related measures: whether sample members worked in 9 of the 12 quarters in years 3 to 5 and 
whether they experienced earnings growth during the follow-up period.^ 

In general, there is little indication that either the employment-focused or the education- 
focused programs improved stable employment or high earnings in year 5. Only three programs 
— Riverside LFA, Riverside HCD, and Portland — resulted in significantly more people work- 
ing in all four quarters of year 5. Likewise, only three programs — Riverside LFA, Columbus 
Integrated, and Portland — resulted in significantly more people earning at least $10,000 in year 
5. Moreover, effects for all programs were relatively small — that is, less than 5 percentage 
points — on these measures. 

A slightly different pattern emerges when stable employment and earnings growth are 
measured over more years of follow-up. (See Appendix Table C.6.) All four employment- 
focused programs, but only two of seven education-focused programs led to an impact on the 
measure of stable employment during years 3 to 5. However, on the measure of earnings over 
time, only the Riverside LFA and Portland programs, along with the Riverside HCD program, 
led to increases over the control group. The Portland program led to the largest impacts on 
both outcomes, which helps explain why it continued to produce earnings impacts in the last 
years of follow-up. 

E. High School Graduates and Nongraduates 

The effects of employment- and education-focused programs may vary according to edu- 
cation credential. Many proponents of skill-building activities have argued that education and 
training are particularly important for people who have not graduated from high school because 
nongraduates typically earn much less than graduates. Proponents of education-focused programs 
have asserted that job search programs will likely be ineffective for people with low skills and 
without education credentials, because these problems prevent many people from finding a job, 
even when assisted by a welfare-to-work program. Moreover, many nongraduates who do find 
employment will likely work at unstable jobs with low salaries and few if any benefits. On the 
other hand, people who had not graduated from high school may not attend long enough to bene- 
fit from the additional education provided in the education-focused programs; or, alternatively, 
local employers may have little demand for the skills they acquired. 

There is also debate over whether employment- or education-focused programs provide 
greater benefit to enrollees who had already attained a high school diploma or GED certificate be- 
fore entering a welfare-to-work program. According to one argument, many high school graduates 
have previous work experience and only require assistance in finding a job, or perhaps a referral to 
a better job than they could have found on their own. In these instances, employment-focused pro- 
grams (particularly the job-search-first programs) should lead to larger effects than education- 
focused programs, which require people to forgo work. However, high school graduates in educa- 
tion-focused programs often take college courses or enroll in vocational training. If these types of 



’in addition, Freedman, 2000, describes results on employment stability and earnings growth through four years 
in much greater detail, and Michalopoulos, 2001, compares the effects of the NEWWS programs with effects from 
programs in Minnesota and Canada that supplemented earnings to encourage work. 





education and training provide valuable skills, then education-focused programs may lead to more 
substantial effects for high school graduates than employment-focused programs. 

The relative success of Portland’s employment-focused program that used a mix of job 
search and education depends largely on its ability to determine who would benefit from educa- 
tion and training and who would benefit from job search. If identification of the two groups was 
quite successful, then the program might have had the largest effects over a long period for both 
high school graduates and nongraduates. It would have been able to increase earnings quickly for 
those capable of finding good jobs quickly and increase earnings later for those who would not 
have found good jobs quickly but who could have done so after short-term skills training. 

Figure 4.3 shows how much the programs benefited high school graduates and nongradu- 
ates by showing the effects of the 1 1 programs on earnings over five years for these subgroups. 
(The next section of this chapter provides a more detailed comparison of the LFA and HCD pro- 
grams for graduates and nongraduates.) 

Only 4 of the 10 programs significantly increased earnings for high school graduates. 
While 3 of these 4 programs were employment-focused — suggesting that job search is more ef- 
fective for this group — there were no large differences in either Atlanta or Grand Rapids in the 
effect of the LFA and HCD programs for graduates. Moreover, the largest impact on total earn- 
ings for high school graduates was in Portland, which allowed some high school graduates to ob- 
tain education and training and which resulted in a substantial increase in the use of vocational 
training and post-secondary education by high school graduates. 

Only 5 of the 1 1 programs significantly increased earnings for nongraduates. Again, results 
were more positive for employment-focused programs than for education-focused programs. Three 
of the 4 employment-focused programs significantly raised earnings for nongraduates compared 
with only 2 of the 7 education-focused programs. Moreover, in each site the LFA program had a 
larger effect on earnings than the HCD program. However, the Portland employment-focused pro- 
gram, which used a mix of initial activities, had the largest effect for nongraduates. 

There is also no clear indication that the welfare-to-work programs as a whole were more 
effective for one group or the other." Five of the programs had larger effects for high school 
graduates than for nongraduates, but five had larger effects for nongraduates. These differences 
appear to be related more to site than to program approach. For example, both Atlanta programs 
had larger effects for high school graduates than for nongraduates, but both Grand Rapids pro- 
grams had larger effects for high school nongraduates than for graduates. 



'**The few high school graduates who were assigned to the Riverside HCD program were considered in need of 
basic education and included in the group of nongraduates shown. 

"Michalopoulos and Schwartz, 2001, report that impacts over a three-year follow-up period were larger for high 
school graduates than for nongraduates. Among the programs they studied, three of the four with the largest differ- 
ences in earnings impacts between high school graduates and nongraduates were in the Alameda, Riverside, and San 
Diego GAIN programs, all of which used a mix of initial activities and none of which were in the NEWWS Evalua- 
tion. Even among the 20 programs that they studied, however, impacts were larger for high school graduates in 1 1 
programs but larger for nongraduates in 9 programs. 



O 

ERIC 



- 101 - 



146 



Earnings gains (S) Earnings gains (S) 



National Evaluation of Welfare- to-Work Strategies 
Figure 4,3 

Impacts on Total Earnings in Years 1 to 5 for Sample Members 
With and Without a High School Diploma or GED at Random Assignment 
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NOTES: See Appendix A. 1. 
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IV. Comparing Results for the LFA and HCD Programs 



In comparing effects of the 1 1 programs, the section above provided informal compari- 
sons of job search and education as alternative approaches to encouraging employment and in- 
creasing earnings. Fortunately, NEWWS was set up to allow a more formal comparison of the 
two approaches. In Atlanta, Grand Rapids, and Riverside (for those in need of basic education), 
people were assigned at random to either the LFA program, which required most participants to 
look for work intially, or the HCD program, which required most to enroll in education or train- 
ing initially. Because people were assigned at random to the two programs, the difference in out- 
comes between individuals in the two programs provides a reliable indication of which approach 
was more effective. Table 4.3 makes this comparison for number of quarters employed and earn- 
ings over the five-year follow-up period. 

For the full sample, Atlanta and Grand Rapids LFAs worked and earned about as much as 
HCDs.'^ For example, in Atlanta LFAs worked 8.5 quarters on average during the 20-quarter fol- 
low-up period and HCDs worked 8.3 quarters. Likewise, people in both programs earned be- 
tween $19,000 and $20,000 on average. 

Any significant differences between the two programs were in favor of the LFA approach. 
In particular, in Grand Rapids the LFA group worked 9.8 quarters on average and the HCD group 
worked 9.5 quarters. Other differences between outcomes for the full samples in the two sites 
were not statistically significant; however, employment and earnings were always higher in the 
LFA programs. 

For people who had not graduated from high school, the results are clearer: Nongraduates 
in the LFA programs were more likely to work than their counterparts in the HCD programs. In 
all three sites the LFA group also had higher average earnings than the HCD group. Although the 
difference in earnings between the LFA and HCD groups was statistically significant for high 
school nongraduates only in Grand Rapids, the average difference across the three sites was sta- 
tistically significant at the 1 percent significance level. 

For high school graduates, the two program approaches had virtually the same effect. For 
example, sample members worked about 9.5 quarters on average in both Atlanta programs and 
about 10.6 quarters on average in both Grand Rapids programs. There were similarly small dif- 
ferences in earnings in both sites. 

Although the job-search-first approach was sometimes more effective than the education- 
first approach at encouraging work and increasing earnings, it is important to keep in mind that 
the most effective program — by far — among the NEWWS programs was Portland, a program 
that used both job search and education and training with a focus on employment. Portland’s 
success implies that a “one size fits all” approach is probably not the best approach and that using 
both job search and education may be better if sites can find strategies that can determine which 
individuals within a group might benefit most from school. 



‘^Recall that people not in need of basic education were not randomly assigned to the Riverside LFA program, 
so that a comparison of the two approaches could not be made for the full sample in Riverside. 
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Table 4.3 

LFA-HCD Differences in Total Employment and Earnings in Years 1 to 5 



Site and Program 


Sample 

Size 


LFA 


HCD 


Difference 

(Impact) 


p-Value 






Average number of Quarters emoloved 




Full impact sample 


Atlanta 


2,936 


8.5 


8.3 


0.2 


0.34 


Grand Rapids 


3,099 


9.8 


9.5 


0.4 ♦ 


0.07 


Riverside 


4,980 


6.8 


n/a 


n/a 


n/a 


No high school diploma or GED 


Atlanta 


1,190 


7.0 


6.4 


0.7 ** 


0.04 


Grand Rapids 


1,268 


8.7 


7.8 


0.9 *** 


0.00 


Riverside 


3,182 


6.0 


5.5 


0.5 ** 


0.02 


High school diploma or GED 


Atlanta 


1,742 


9.5 


9.6 


-0.1 


0.73 


Grand Rapids 


1,827 


10.6 


10.6 


0.0 


0.96 


Riverside 


1,798 


7.8 n/a n/a 

Total earnings in years 1 to 5 


n/a 


Full impact sample 


Atlanta 


2,936 


19,838 


19,397 


442 


0.59 


Grand Rapids 


3,099 


22,323 


21,616 


706 


0.39 


Riverside 


4,980 


17,438 


n/a 


n/a 


n/a 


No high school diploma or GED 


Atlanta 


1,190 


13,439 


12,344 


1,095 


0.26 


Grand Rapids 


1,268 


16,243 


14,299 


1,945 ** 


0.04 


Riverside 


3,182 


13,193 


12,273 


920 


0.18 


High school diploma or GED 


Atlanta 


1,742 


24,163 


24,111 


. 52 


0.97 


Grand Rapids 


1,827 


26,496 


26,729 


-233 


0.85 


Riverside 


1,798 


23,019 


n/a 


n/a 


n/a 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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V. Effects on Employment and Earnings Impacts bv the End 
of the Control Group Embargo 

This section considers whether the NEWWS Evaluation underestimated the longer-term 
effects on employment and earnings in sites where the embargo on control group services ended 
after year 3. It concludes that the effect of removing the embargo was likely to be small except in 
Portland, but that most programs’ effects on employment and earnings would likely have dimin- 
ished in years 4 and 5 even if the control group embargo had remained in place. Because the em- 
bargo did appear to have an effect in Portland, results in this report were limited in Portland to 
499 control group members who were prohibited from receiving program services for five years. 

The likely effect of ending the control group embargo was examined in four sites where 
some control group members gained access to program services in year 4 and some did not gain 
access until later. In those four sites — Atlanta, Columbus, Grand Rapids, and Portland — con- 
trol group members were divided into a “short-term embargo” group whose members became 
eligible for program services in year 4 and a “longer-term embargo” group, whose members re- 
mained ineligible for program services for at least four years. 

These comparisons were not made in Riverside, Oklahoma City, and Detroit. In River- 
side, the test was unnecessary because all control group members remained ineligible for ser- 
vices for five years after random assignment. In Oklahoma City, the fact that the program did 
not significantly affect employment or earnings during years 1 to 3 — when all control group 
members remained ineligible for services — suggests that ending the embargo did not alter the 
program’s effects. In Detroit, the test could not be performed because all control group mem- 
bers became eligible for services after three years of follow-up. It is possible that the ending 
the embargo caused impacts in the program to decline after year 3, but there is no way to know 
whether that was true. 

In comparing the short-term and longer-term embargo groups, the rate of decline of im- 
pacts for the two groups in year 4 and the size of impacts for the longer-term embargo group in 
year 4 were examined. If impacts declined faster for the short-term embargo group than for the 
longer-term embargo group in year 4, it is evidence that ending the embargo did reduce impacts. 
In contrast, if a program did not significantly affect outcomes for the longer-term embargo group 
in year 4 — even though the group was still embargoed from receiving services — it is evidence 
that ending the embargo did not have a large effect because the effectiveness of the program 
ended before the embargo. 

Portland provided the most reliable comparison. Several years into the Portland evalua- 
tion, 499 control group members were chosen at random from the control group and remained 
ineligible for services for five years after random assignment. This group of 499 constituted the 
longer-term embargo group while the short-term embargo group consisted of all other control 
group members. Because the groups were chosen at random, they did not differ systematically 
from each other in background characteristics, and both groups resembled the program group. 

A comparison of the short-term and longer-term embargo groups in Portland suggested 
that ending the control group embargo did contribute to a decline in impacts there. The effect of 
the program on employment was about the same in year 3 whether it was calculated with the 
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short-term or longer-term embargo group (12 versus 11 percentage points). In year 4, however, 
the effect was only 2.6 percentage points when measured with the short-term embargo group but 
8.5 percentage points when measured with the longer-term embargo group. In year 5, the effect 
had largely disappeared when measured with the short-term embargo group but was 3.8 percent- 
age points when measured with the longer-term embargo group. Differences in the trends in earn- 
ings impacts were similar to differences in the trends in employment impacts during years 3 to 5. 
Because of these differences, MDRC decided to include only members of the longer-term em- 
bargo control group in estimating impacts of the Portland program. 

Comparisons were conducted less rigorously in the other three sites. Control group mem- 
bers in the short-term and longer-term embargo groups were randomly assigned at different times 
and therefore might have differed systematically from each other at the time of random assign- 
ment. Consequently, one cannot rule out the possibility that variation in impacts between the two 
groups resulted from differences in background characteristics, labor markets, or other external 
factors that may have affected employment. 

Despite these measurement problems, it appears that ending the control group embargo 
was not responsible for the small effect on employment and earnings in Grand Rapids at the end 
of the follow-up period. For both programs, the effect on employment and/or earnings declined 
during year 3, when all control group members were ineligible for services. This downward trend 
in impacts continued in years 4 and 5 and was seen in both the short-term and longer-term co- 
horts for both the LFA and the HCD programs. 

In Atlanta, the trend in impacts after year 3 suggests that the end of the control group em- 
bargo had only a limited effect. For the longer-term embargo group, the effect of the LFA pro- 
gram on employment and earnings was no longer statistically significant by the second half of 
year 4 even though control group members were not yet eligible for program services. In other 
words, it appears that the effectiveness of the program had largely disappeared before control 
group members were allowed to receive program services. A similar pattern was seen for the 
HCD program in Atlanta.'^ 

The evidence is similar for the Columbus Integrated program. During year 3, the esti- 
mated effect of the program on employment was 4.3 percentage points and statistically signifi- 
cant for the longer-term embargo group. In year 4, however, the effect on employment had de- 
creased to 2.4 percentage points and was not statistically significant. The program’s effect on 
earnings also began to decline during year 3 and continued downward slightly during year 4. 

Ending the embargo may have made more of a difference for the Columbus Traditional 
program. Impacts on earnings for the first three years after random assignment were similar 
whether calculated for the short-term or the longer-term embargo groups. In years 4 and 5, how- 
ever, the estimated impact on earnings was about $950 higher when calculated with the longer- 
term embargo group. On the other hand, Columbus Traditional had only a negligible effect on 
employment for either sample, either before or after year 3. 



'^For unknown reasons, the effects of the HCD program were larger in years 4 and 5 when calculated with the 
short-term embargo group than the longer-term embargo group. 
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Whether or not allowing the control group to receive program services diminished the ef- 
fect of the programs at the end of the follow-up period, the essential story is clear from the first 
three years of follow-up: The Portland program had the largest and most consistent effects by far, 
employment-focused job-search-first programs had the next largest effects, and education- 
focused programs had the smallest effects. Moreover, issues surrounding the control group em- 
bargo do not affect the comparison of LFA and HCD programs in Atlanta, Grand Rapids, and 
Riverside. That comparison indicates that the two program approaches were equally effective for 
graduates, that the LFA programs were more effective for nongraduates, and that there was no 
evidence that the cumulative effect of the HCD programs would surpass the cumulative effect of 
the LFA programs with longer follow-up. 



Chapter 5 

Impacts on Public Assistance 



Welfare- to -work programs have two goals: increasing employment and earnings and re- 
ducing welfare receipt and benefit amounts. Chapter 4 discussed which programs were most suc- 
cessful with the work part of the transition. The Portland program had the largest and most per- 
sistent effects. Employment-focused programs increased employment quickly, and both employ- 
ment- and education-focused programs increased employment in the intermediate term. For peo- 
ple who had not graduated from high school, however, job search appeared more effective than 
education at increasing earnings and employment. 

This chapter explores the effects of the programs on welfare and Food Stamp receipt and 
benefit amounts over five years, estimated from automated state and county payment records. In 
addition to showing the effects of the programs on these outcomes for all programs for the full 
report sample, the chapter compares the effects of the programs for high school graduates and 
nongraduates and compares the effects of the LFA and HCD programs in Atlanta, Grand Rapids, 
and Riverside. 

I. Key Findings 

• Over five years, program group members in. all programs spent less time on 
welfare and received smaller welfare payments on average than control group 
members. Welfare savings were larger and more persistent than earnings 
gains: Few programs continued to affect employment and earnings in year 5, 
but most programs continued to generate welfare savings at the end of year 5. 

• Welfare savings were generally larger for programs that had larger effects on 
earnings, but they varied for other reasons as well. Total payments were re- 
duced more in higher-grant sites such as Riverside, Portland, and Grand Rap- 
ids and less in low-grant sites such as Atlanta. Benefits were reduced more in 
sites such as Columbus and Grand Rapids that strictly enforced mandates than 
in sites such as Detroit that did not. 

• Direct comparisons between the LFA and HCD programs showed that LFA 
programs generally resulted in lower welfare use and expenditures. However, 
reductions were found for only some programs: Atlanta and Grand Rapids 
LFA for nongraduates and Grand Rapids LFA for graduates. 

• The programs had similar effects for high school graduates and nongraduates. 

Most programs produced welfare savings for both groups, and there was little 
evidence that the effects were larger for one group than the other: In five pro- 
grams welfare savings were larger for high school graduates and in five pro- 
grams welfare savings were larger for nongraduates. 

• Over five years, program group members in all programs spent less time on 
Food Stamps and received smaller Food Stamp payments on average than con- 
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trol group members. Effects were generally smaller for Food Stamps than for 
welfare payments, however, because some program group members appropri- 
ately continued to receive Food Stamps after they left welfare. 



II. Expected Effects 

How much a welfare-to-work program reduces welfare use will be influenced by a num- 
ber of factors, including how much the program increased employment and earnings, how strictly 
it enforced participation mandates by using sanctions, and the generosity of the state’s welfare 
benefits. 

In all of the NEWWS sites, as family earnings increased, welfare payments decreased. 
Therefore, it is reasonable to expect programs that led to the largest increases in earnings oyer 
five years — particularly Portland — to also have produced the largest decreases in welfare re- 
ceipt and expenditures. 

Sanctions (described in Chapter 1) also have a direct effect on welfare use. As a result, 
tough, enforcement-oriented programs like Grand Rapids HCD could have reduced welfare pay- 
ments much more than they increased employment or earnings. Because the programs studied in 
NEWWS did not use “full-family sanctions,” which remove all of a family’s welfare grant,’ the di- 
rect effect of sanctions would be to reduce payment amounts rather than time on welfare. Neverthe- 
less, a family whose benefits were reduced through sanctions might have decided to leave welfare 
altogether, so that sanctions might have indirectly resulted in fewer people receiving welfare. Re- 
quiring people to participate in welfare-to-work services might also encourage them to leave wel- 
fare even if it does not increase their employment and earnings. For example, some people might 
already be working but not reporting their earnings to the welfare system. Going to school or 
attending a job club might interfere with their ability to perform their job or might be too great a 
burden on top of working, and they might choose to keep their job and stop receiving benefits. 

Differences in maximum welfare grant levels by site and in welfare earnings disregards (de- 
scribed in Chapter 1) could also influence the effects that programs have on welfare use. Other 
things being equal, savings in welfare expenditures will be larger in high-grant states simply be- 
cause there are more dollars to save. On the other hand, reductions in months on welfare will be 
larger in low-grant states because people lose their eligibility for cash assistance at a lower level of 
earnings. Likewise, reductions in both welfare payment amounts and welfare receipt will be smaller 
in states with relatively more generous earnings disregards (that is, those that allow working welfare 
recipients to keep more of their welfare payments), like California and Georgia. 

Site-by-site differences in background characteristics of sample members may also be re- 
lated to program effects on welfare use. Welfare savings may be greater in sites where most sam- 
ple members had long stays on welfare before random assignment or faced other severe barriers 
to employment, because control group members in these sites were likely to remain on assistance 
for a long time. If long-term welfare recipients have severe barriers that keep them from working 
and that are not ameliorated by the programs, however, then sites with a more disadvantaged 



'Pavetti and Bloom, 2001. 
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caseload might have smaller effects on welfare use. (Chapter 7 examines this question by inves- 
tigating whether the programs increased the earnings of long-term welfare recipients and reduced 
their welfare benefit levels.) 

In short, we should expect relatively high reductions in welfare receipt and payment 
amounts in Portland, which increased earnings the most and had relatively generous benefits. We 
might also expect high reductions in Riverside LFA, which had the second highest impact on 
earnings and had the most generous benefits, although California’s generous earnings disregard 
might have diminished the program’s effect on welfare use. We should expect higher reductions 
in welfare use in Grand Rapids and Columbus than would be indicated by their impacts on em- 
ployment and earnings because these sites had some of the toughest sanction policies, but smaller 
reductions in Detroit than would be indicated by their impacts on employment and earnings be- 
cause Detroit did not strictly enforce participation mandates for much of the follow-up period. 
Finally, we should expect relatively small reductions in welfare benefit amounts in Atlanta — 
even though its two programs increased earnings — because it had such low grant levels and had 
a more generous earnings disregard policy than any site except Riverside. 

The effects of the programs on Food Stamp receipt and benefit amounts are harder to pre- 
dict. Regarding cash assistance, as earnings increase. Food Stamp benefits decrease. However, 
additional earnings generally reduce Food Stamps less than welfare: Each additional dollar of 
earnings reduces Food Stamp amounts by less than a dollar. In addition. Food Stamp grant calcu- 
lations count a dollar of earnings less than a dollar of welfare, so a person who replaces welfare 
dollars with earnings may experience a net increase in Food Stamps.^ It is also possible that re- 
cipients gave up Food Stamps after they left welfare for employment or other reasons, even if 
they still qualified for them, either because they wanted to leave public assistance entirely or be- 
cause they did not know they were still eligible for noncash assistance. 

III. Welfare Receipt and Payments 

A. Welfare Receipt and Payments Over Five Years 

Table 5.1 shows the average number of months that program and control group members 
received cash assistance and the payment amounts they received on average. Both of these out- 
comes were measured over the five-year follow-up period from administrative records collected 
from state and county welfare systems. The table also shows the impact of the programs (as al- 
ways, measured as the difference in average outcome levels between the program and control 
groups) and their levels of statistical significance.^ Because welfare administrative records were 
available in Oklahoma City for only three years, results for Oklahoma City are not shown in most 
tables and figures in this chapter. 



^The Food Stamp benefit level equals the maximum benefit level minus 30 percent of a household’s countable 
income. Countable income includes welfare payments plus 80 percent of earnings, so a sample member who replaces 
welfare with earnings could lower her countable income and thus increase her Food Stamp payments (Ohls and Bee- 
bout, 1993). 

^Appendix Table D.l shows results on the same outcomes for the first three years of follow-up. Appendix Table 
D.2 shows results on welfare receipt for the last quarter of each follow-up year. 
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National Evaluation of Welfare-to-Work Strategies 
Table 5.1 



Impacts on Welfare Receipt and Payments in Years 1 to 5 



Site and Program 


Sanq^le 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Percentage 
Change (%) 




Average number of months of welfare receipt in years 1 to 5 


Atlanta Labor Force Attachment 


2,938 


34.4 


37.2 


-2.9 *** 


-7.7 


Atlanta Human Capital Development 


2,992 


35.3 


37.2 


-1.9 *** 


-5.2 


Grand Rapids Labor Force Attachment 


3,012 


26.9 


31.1 


-4.2 *** 


-13.6 


Grand Rapids Human Capital Development 


2,997 


28.2 


31.1 


-2.9 *** 


-9.2 


Riverside Labor Force Attachment 


6,726 


27.8 


31.0 


-3.2 


-10.5 


Lacked high school diploma or basic skills 


3,125 


30.1 


33.3 


-3.2 *** 


-9.6 


Riverside Human Capital Development 


3,135 


30.0 


33.3 


-3.3 *** 


-9.8 


Columbus Integrated 


4,672 


23.3 


27.2 


-3.9 *** 


-14.4 


Columbus Traditional 


4,729 


24.7 


27.2 


-2.5 *** 


-9.2 


Detroit 


4,459 


36.1 


37.7 


-1.6 *** 


-4.3 


Oklahoma City 


n/a 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


19.8 


25.3 


-5.6 *** 


-21.9 




Average total welfare payments received in years 1 to 5 fS) 


Atlanta Labor Force Attachment 


2,938 


9,064 


9,946 


-881 *** 


-8.9 


Atlanta Human Capital Development 


2,992 


9,236 


9,946 


-710 *** 


-7.1 


Grand Rapids Labor Force Attachment 


3,012 


10,414 


12,966 


-2,552 *** 


-19.7 


Grand Rapids Human Capital Development 


2,997 


11,199 


12,966 


-1,767 *** 


-13.6 


Riverside Labor Force Attachment 


6,726 


15,584 


18,294 


-2,710 *** 


-14.8 


Lacked high school diploma or basic skills 


3,125 


17,171 


20,126 


-2,955 *** 


-14.7 


Riverside Human Capital Development 


3,135 


17,176 


20,126 


-2,949 *** 


-14.7 


Columbus Integrated 


4,672 


7,481 


9,005 


-1,523 *** 


-16.9 


Columbus Traditional 


4,729 


7,899 


9,005 


-1,105 *** 


-12.3 


Detroit 


4,459 


15,686 


16,247 


-561 ** 


-3.5 


Oklahoma City 


n/a 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


8,940 


11,686 


-2,746 *** 


-23.5 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES : See Appendix A. 1 . 
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Table 5.1 shows that all 10 programs in which welfare payments were available had a 
significant effect on the number of months that people received welfare. The effect ranged from 
1.6 months (over a five-year, or 60-month, period) in Detroit to 5.6 months in Portland. All 10 
programs also reduced welfare payments over the five-year period (relative to control group lev- 
els), with welfare savings ranging from $561 in Detroit to $2,949 in Riverside HCD (although 
the percentage change was largest in Portland: 23 percent). 

Results by site are generally consistent with the expectations outlined in the prior section. 
First, the program with the largest effects on employment and earnings — Portland — also had the 
largest reduction in welfare use. These reductions are especially impressive considering the Port- 
land control group’s relatively low welfare use. On average, control group members in Portland re- 
ceived welfare for about 25 months over the 60-month period. In comparison, program group 
members in Portland received welfare for a little less than 20 months, for a reduction of 5.6 months, 
or nearly half a year diuing the five-year follow-up period. Portland also had the second-largest re- 
duction in dollars spent on welfare — an average of $2,746 per program group member. 

Second, the site with the most generous grant amounts — Riverside — also generated rela- 
tively large welfare savings.'* Over the five-year period, HCD program group members received 
nearly $3,000 less than control group members on average while LFA program group members re- 
ceived about $2,700 less than control group members.^ The fact that the two programs reduced 
benefits by similar amounts is somewhat surprising considering that the effect of the LFA program 
on earnings was twice as large as the effect of the HCD program. This finding suggests that the 
HCD program discouraged a number of people from receiving benefits without helping them get a 
job. If that occurred, the HCD program will have left individuals with less total income on average 
than they would have had without the program, an issue addressed in Chapter 6. 

Third, reductions in welfare use were fairly large in programs that had tough sanction 
policies (Columbus Integrated and both programs in Grand Rapids) and fairly small in the site 
with the least strict sanction policy (Detroit). This comparison is especially revealing because 
Detroit and Grand Rapids, which are both in Michigan, had similar grant levels and earnings dis- 
regards and because the effect on earnings over five years was about as large in Detroit as in 
Grand Rapids LFA and nearly twice as large as in Grand Rapids HCD. Detroit had the smallest 
reductions in payment amounts ($561 over five years) and months on welfare (L6 months on av- 
erage). In comparison. Grand Rapids LFA reduced time on welfare by 4.2 months and average 
welfare payments by more than $2,500, and Grand Rapids HCD reduced time by 2.9 months and 
payments by $1,800. 

Finally, reductions in welfare use were relatively small for both programs in Atlanta. 
Time on welfare was reduced by 2.9 months in the LFA program and 1.9 months in the HCD 
program, while cash payments over five years were reduced by about $900 in the LFA program 
and $700 in the HCD program. Decreases in welfare were smaller than in Grand Rapids or Riv- 
erside, even though the effects on earnings in Atlanta rivaled those in the other two sites. The 

‘‘Welfare spending was also the highest for control group members in Riverside. The notion that savings might 
be greater in high-grant states does not mean that a state should raise its welfare grant to generate savings. 

^As shown in Table 5.1, Riverside LFA and HCD led to similarly large reductions in welfare expenditures for 
sample members determined to need basic education. 
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relatively low level of welfare savings in Atlanta may reflect the low level of benefits available; 
people who lost welfare when they went to work could not lose very much.^ In addition, the 
site’s use of fill-the-gap budgeting allowed working welfare recipients to keep relatively more of 
their grant than they could in the other sites. 

B. Welfare Receipt by Year 

As discussed above, there are two direct causes of welfare savings: increased earnings 
and use of sanctions. Chapter 4 showed that program services had immediate effects in the em- 
ployment-focused programs, but delayed effects in the education-focused programs. If earnings 
were primarily responsible for reductions in welfare use, then reductions in welfare use should 
also have been larger initially in employment-focused programs than in education-focused pro- 
grams (particularly in the same sites). Although sanctions did not stop people from receiving 
welfare in any of the sites, sanctions may have given people a reason to voluntarily stop receiving 
benefits. If reductions in welfare receipt were due to sanctions (indirectly) or to people voluntar- 
ily leaving welfare rather than complying with programs requirements, welfare savings should 
have been fairly close initially in the two types of programs. 

Figure 5.1 and Appendix Table D.2 show the proportion of program and control group 
members who received welfare in the last quarter of each of the five years of follow-up. By 
showing results by year, the figure and table also can be used to examine results for the first three 
years, before any control group members were allowed to receive program services. They also 
indicate how quickly people left welfare and how many remained on welfare at the end of year 5 
— a number that may be important in a world of time-limited welfare. 

According to Figure 5.1 and Appendix Table D.2, many people left welfare in each site, 
even in the absence of welfare-to-work services. In Atlanta, as discussed in earlier chapters, vir- 
tually everyone was receiving welfare at the time of random assignment (that is, the quarter prior 
to year 1). By the end of year 1, however, about 83 percent of control group members were still 
on welfare. The decline in welfare receipt continued throughout the follow-up period. By the last 
quarter of year 5, less than 40 percent of the control group in Atlanta still received welfare. 

Trends in program impacts (that is, the difference in welfare receipt between the program 
and control groups) suggest that increased employment and earnings were primarily responsible 
for reductions in welfare receipt — at least during the early years of follow-up. The two Atlanta 
programs illustrate the basic points.^ At the end of year 1, Atlanta LFA significantly reduced wel- 
fare receipt, but Atlanta HCD did not. This is consistent with the finding in Chapter 4 that the 
LFA program had a larger initial effect on employment and earnings than the HCD program. 
Moreover, the magnitude of the impacts on welfare receipt at the end of the first year were simi- 
lar to the impacts on employment shown in Figure 4. 1 . 



*An alternative possibility is that Atlanta’s low grants meant that control group members left welfare quickly, 
which was not the case. The average control group member in Atlanta received welfare for 37 months during the 
five-year follow-up period, longer than in all other sites except Detroit. 

’The Atlanta programs are discussed here because they appear side by side on the figure and used different self- 
sufficiency approaches. Section V of this chapter explicitly compares the LFA and HCD programs in Atlanta, Grand 
Rapids, and Riverside and indicates where the programs resulted in statistically significant differences in outcomes. 
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National Evaluation of Welfare-to-Work Strategies 
Figure 5.1 

Impacts on Welfare Receipt in the 
Last Quarter of Years 1 to 5 
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Figure 5.1 (continued) 
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Figure 5.1 (continued) 
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SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1. 
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The impact of Atlanta HCD on welfare receipt increased after year 1 and was statistically 
significant in years 2 and 3. This trend paralleled Atlanta HCD impacts on employment and earn- 
ings. (See Chapter 4.) Similarly, Atlanta LFA increased employment and earnings above control 
group levels in years 2 and 3 and continued to decrease welfare receipt during these years. These 
patterns suggest that reductions in welfare receipt were driven largely by increases in employ- 
ment and earnings. 

Impacts of the two Atlanta programs on welfare receipt declined toward the end of the 
follow-up period. However, reductions continued even after these programs ceased to raise em- 
ployment levels above the control group. In particular, Atlanta LFA continued to significantly 
reduce welfare receipt at the end of year 5, even though it did not significantly increase employ- 
ment during year 5. Likewise, Atlanta HCD significantly reduced welfare receipt at the end of 
year 4, even though it did not significantly increase employment during year 4. 

The trends for the other employment-focused and education-focused programs were simi- 
lar to the patterns for the Atlanta programs. First, many people left welfare in each site, even in 
the absence of welfare-to-work services. Control group merhbers left the welfare rolls particu- 
larly fast in Columbus and Portland. In both sites, fewer than 20 percent of control group mem- 
bers were still receiving welfare at the. end of year 5. In comparison, about 35 percent of control 
group members in Atlanta, Detroit, and Riverside remained on welfare. 

The trends in impacts were also similar to the patterns for the two Atlanta programs. Dur- 
ing the first year, the employment-focused programs led to larger decreases in welfare receipt 
than the education-focused programs. All four employment-focused programs significantly re- 
duced welfare receipt, and three of these programs (Grand Rapids and Riverside LFA and Port- 
land) produced the largest decreases (of 7 to 9 percentage points) among the 1 1 programs. In con- 
trast, five of the seven education-focused programs reduced welfare receipt in year 1, and de- 
creases ranged from 1 to 4 percentage points for most programs.* 

Like Atlanta HCD, the other education-focused programs led to larger reductions in wel- 
fare receipt after year 1 . This trend was most noticeable in Detroit, where impacts were not statis- 
tically significant in year 1 but became statistically significant in years 2 and 3 (just as Detroit’s 
effects on employment and earnings became statistically significant in years 2 and 3). 

Like Atlanta LFA, the other employment-focused programs continued to reduce welfare 
receipt in years 2 and 3, a trend that (for the most part) mirrored their effects on employment and 
earnings. For example. Riverside LFA impacts on welfare receipt declined only slightly between 
years 1 and 2, just as its effects on earnings declined slightly, \yhile Portland impacts on welfare 
receipt increased between years 1 and 2, just as its effects on employment increased somewhat 
between the two years.^ 



*The small program-control group difference (of 2 percentage points) for Atlanta LCD had a p-value of .014, 
just above the 10 percent level for statistical significance. 

^As discussed above, impacts on welfare receipt may be larger or smaller than expected, given a program’s im- 
pact on employment and earnings. For instance. Riverside LFA employment impacts decreased sharply between 
years 1 and 2, but reductions in welfare receipt changed very little. Similarly, Portland impacts on total earnings in- 
creased dramatically between years 1 and 2, but reductions in welfare receipt increased to a lesser extent 
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The impacts on welfare receipt decreased over time in all of the programs, but most pro- 
grams continued to significantly reduce welfare receipt at the end of year 5. This result is some- 
what surprising, given that few programs increased employment and earnings above control 
group levels in year 5. This pattern is especially striking for Grand Rapids LFA, which decreased 
receipt below the control group by 3 percentage points at the end of year 5, but led to a similar 
reduction in percentage employed during that year. This finding implies that some program 
group members who exited welfare for employment earlier in the follow-up did not return to as- 
sistance after leaving employment, even though they were eligible to, a pattern that may reflect 
the national climate in the aftermath of the federal welfare reform legislation of 1996. 

C. Comparing Hi£h School Graduates and Noneraduates 

As discussed in Chapter 4, employment- and education-focused programs were expected 
to have different effects on employment and earnings depending on whether people were job- 
ready or in need of more education. There are also reasons to expect welfare-to-work programs to 
produce different amounts of welfare savings for the two groups. High school nongraduates re- 
ceived higher welfare benefits on average than graduates, probably because they have a harder 
time finding work.'° Welfare savings may therefore be larger for high school nongraduates than 
for graduates. When they do go to work, however, graduates are likely to earn more than non- 
graduates, and are probably more likely to find jobs that pay enough to help them leave welfare. 
A program that is just as likely to help high school graduates as nongraduates find work may' 
therefore produce larger welfare savings for graduates. 

Figure 5.2 explores these possibilities by comparing program effects on welfare benefits 
over five years separately for high school graduates and nongraduates. According to the figure, 
there is little evidence that the programs reduced benefits more for one group than the other (just 
as there was little evidence in Chapter 4 that the programs were systematically affecting earnings 
more for one group than the other). Over five years, virtually all of the programs reduced welfare 
payments for both subgroups. Moreover, in five of the nine programs where effects could be 
measured over five years, they were larger for high school nongraduates than graduates (both 
Grand Rapids programs, both Columbus programs, and Riverside LFA), but in the other four 
programs effects were larger for graduates than for nongraduates (both Atlanta programs, Detroit, 
and Portland). 

IV. Food Stamp Payments and Receipt 

Table 5.2 examines the effects of the programs on the number of months that sample 
members received Food Stamps and Food Stamp expenditures for the five-year follow-up period. 
(Appendix Table D.3 shows results on the same outcomes for the first three years of follow-up, 
which is the period before any control group members could have received program services. 
Appendix Table D.4 shows results for the last quarter of year 5.) 



*®Michalopoulus and Schwartz, 2001. Among control group members in the NEWWS sites, the proportion of 
high school graduates and GED recipients who ever worked for pay during years 1 to 5 exceeded the proportion of 
nongraduates by 3 to 1 1 percentage points (results not shown). 
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National Evaluation of Welfare-to-Work Strategies 
Figure 5.2 

Impacts on Total Welfare Payments in Years 1 to 5 for Sample Members 
With and Without a High School Diploma or GED at Random Assignment 
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SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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National Evaluation of Welfare-to-Work Strategies 
Table 5.2 



Impacts on Food Stamp Payments and Receipt 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Percentage 
Change (%) 




Average number of months of Food Stamp receipt 
in Years 1 to 5 


Atlanta Labor Force Attachment 


2,938 


42.1 


43.4 


-1.4 ** 


-3.2 


Atlanta Human Capital Development 


2,992 


42.2 


43.4 


- 1.2 ** 


- 2.8 


Grand Rapids Labor Force Attachment 


3,012 


30.5 


33.8 


-3 3 *** 


-9.9 


Grand Rapids Human Capital Development 


2,997 


31.7 


33.8 


^2 \ ♦♦♦ 


- 6.2 


Riverside Labor Force Attachment 


6,726 


25.3 


29.0 


5 ♦♦♦ 


- 12.6 


Lacked high school diploma or basic skills 


3,125 


27.7 


31.2 


5 ♦♦♦ 


- 11.2 


Riverside Human Capital Development 


3,135 


27.4 


31.2 


-3 8 *** 


-12.3 


Columbus Integrated 


4,672 


27.9 


31.2 


-3 4 *** 


-10.7 


Columbus Traditional 


4,729 


29.1 


31.2 


^2 \ ♦♦♦ 


- 6.6 


Detroit 


4,459 


38.8 


40.3 


^ 414141 


-3.8 


Oklahoma City 


n/a 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


29.4 


32.5 


,3 J 4«4«4« 


-9.4 




Average total Food Stamps received in vears 1 to 5 fS) 


Atlanta Labor Force Attachment 


2,938 


10,661 


11,089 


-428 ** 


-3.9 


Atlanta Human Capital Development 


2,992 


10,930 


11,089 


-159 


-1.4 


Grand Rapids Labor Force Attachment 


3,012 


6,351 


6,966 


-615 *** 


- 8.8 


Grand Rapids Human Capital Development 


2,997 


6,580 


6,966 


-387 *** 


-5.6 


Riverside Labor Force Attachment 


6,726 


4,981 


5,870 


-888 ♦♦♦ 


-15.1 


Lacked high school diploma or basic skills 


3,125 


5,577 


6,504 


-928 *** 


-14.3 


Riverside Human Capital Development 


3,135 


5,492 


6,504 


-1,013 ♦♦♦ 


-15.6 


Columbus Integrated 


4,672 


7,160 


8,185 


-1,025 ♦♦♦ 


-12.5 


Columbus Traditional 


4,729 


7,537 


8,185 


-648 ♦♦♦ 


-7.9 


Detroit 


4,459 


9,186 


9,519 


-334 ♦♦ 


-3.5 


Oklahoma City 


n/a 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


6,926 


7,753 


-827 *** 


-10.7 



SOURCE: MDRC calculations from state and county administrative records. 



NOTES: See Appendix A. 1 . 



O 

ERIC 



- 120 - 



165 



In all sites except Riverside, control group members received Food Stamps for more months 
than they received welfare payments. Most likely, some control group members in these sites re- 
ceived too much income from earnings or other sources to receive cash assistance but remained eli- 
gible for Food Stamps. The difference between Food Stamp and welfare receipt was largest in Port- 
land, where control group members received Food Stamps for 32 months on average during years 1 
to 5, while they received welfare for only about 25 months.” Similarly, in Atlanta, which is the 
only site where the federal maximum Food Stamp allotment exceeded the maximum welfare grant, 
control group members received Food Stamps for an average of 43 months over five years (the 
most among the six sites), about six months longer than they received welfare. 

The importance of Food Stamps as an income supplement for control group members 
varied from place to place. These differences largely reflect the generosity of welfare benefits and 
Food Stamp rules, which reduce Food Stamps by a certain amount for each dollar of welfare 
benefits. At one extreme, Atlanta’s maximum welfare grant is so low that people received more 
on average from Food Stamps — about $11,000 from Food Stamps compared with nearly 
$10,000 from welfare over the five-year period. In Riverside, in contrast, control group members 
received less than $6,000 in Food Stamp benefits on average, compared with welfare benefits of 
$18,000. Over five years, all programs significantly reduced total months of Food Stamp receipt 
and all but one program — Atlanta HCD — significantly reduced Food Stamp expenditures. Al- 
though impacts were smaller for Food Stamps than for welfare, programs that reduced welfare 
the most also tended to be the programs that reduced Food Stamp use the most. For example, five 
programs reduced welfare receipt by more than three months: Portland, Grand Rapids LFA, both 
Riverside programs, and Columbus Integrated. (See Table 5.1.) The same five programs reduced 
Food Stamp receipt by more than three months. Likewise, Atlanta HCD and Detroit reduced both 
welfare and Food Stamp receipt by less than two months. Because of the complex interaction be- 
tween welfare and Food Stamp payment amounts, the extent to which programs reduced Food 
Stamp payment amounts below control group levels was less clear-cut. For example, the Colum- 
bus Integrated program, which had effects on welfare benefits in the midrange of the programs 
being studied, had the largest reduction in Food Stamp payments. However, the programs that 
reduced welfare payments the most — Portland and the two Riverside programs — also had 
among the largest effects on Food Stamp payments over five years. 

V. Comparing the Effects of the LFA and HCD Programs 

As discussed in earlier chapters, NEWWS was set up to allow a rigorous comparison of 
LFA and HCD programs in Atlanta, Grand Rapids, and Riverside (for those in need of basic edu- 
cation). According to Chapter 4, differences were concentrated among high school nongraduates, 
for whom LFA programs generally resulted in higher employment and earnings than HCD pro- 
grams (see Table 4.3). 



"Furthermore, in the last quarter of year 5, twice as many control group members in Portland (34.0 percent ver- 
sus 17.1 percent) received Food Stamps as were receiving welfare. In addition, more than one-half of control group 
members in Atlanta were still receiving Food Stamps in the last quarter of year 5 compared with a little more than 
one-third who received a welfare check. See Appendix Tables D.2 and D.4. 
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Table 5.3, which shows the effects of the two self-sufficiency approaches on welfare re- 
ceipt and benefit amounts, tells a similar story. In some sites, there was virtually no difference 
between the two approaches. Where there were differences, however, the LFA programs resulted 
in lower welfare use than the HCD programs over five years. 

For the full sample. Grand Rapids LFAs received welfare for an average of just under 27 
months compared with just over 28 months for Grand Rapids HCDs. Although this difference 
was fairly small, it was statistically significant and produced a statistically significant welfare 
savings of nearly $800 per person. Atlanta LFAs were also less likely to receive welfare than At- 
lanta HCDs and received slightly less in welfare payments, but these differences were not statis- 
tically significant. 

At the end of year 5 (not shovm in the table) the LFA programs in Atlanta and Grand 
Rapids continued to have lower welfare use than the HCD programs, although the differences 
were not statistically significant. Thus, HCD programs are unlikely to produce lower cumulative 
welfare use than the LFA programs with a longer follow-up period. 

In Atlanta and Grand Rapids, results for high school nongraduates were more consistent 
than results for the full sample. In these two sites, nongraduates in the LFA program received 
significantly less in welfare payments than nongraduates in the HCD program. However, for 
nongraduates in Riverside, LFAs and HCDs received welfare benefits for the same number of 
quarters and averaged the same amount of welfare payments over five years. Still, when averaged 
across the three sites, LFAs received nearly $500 less in welfare payments over five years than 
HCDs (($531+$905+$6)/3; the difference was statistically significant). Likewise, a simple aver- 
age across the three sites indicates that the high school nongraduates in the LFA programs re- 
ceived welfare for 1.3 months less on average than nongraduates in the HCD programs, a differ- 
ence that was again statistically significant. 

For high school graduates, the LFA program produced greater welfare savings in Grand 
Rapids, but not in Atlanta. In Grand Rapids, LFAs received welfare for an average of 24.3 
months and HCDs for 25.6 months. Although the effect was fairly small, the difference was sta- 
tistically significant, and it led to statistically significant welfare savings of more than $700 per 
person. In contrast, high school graduates in the two Atlanta programs were about equally likely 
to receive welfare and received similar benefit amounts on average. 

Although the job-search-first approach sometimes resulted in less welfare use than the 
education-first approach, it is important to keep in mind that (for the full sample) the greatest 
proportional savings among the NEWWS programs was in Portland, which also generated the 
largest gains in employment and earnings. The fact that Portland used job search, education, and 
training with a focus on employment suggests that the combination of approaches is better than 
either one alone. 
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National Evaluation of Welfare-to-Work Strategies 
Table 5.3 



LFA-HCD Differences in Total Welfare Payments 
and Receipt in Years 1 to 5 



Site and Program 


Sample 

Size 


LFA 


HCD 


Difference 

(Impact) 


p- Value 




Total welfare payments received in years 1 to 5 (S) 


Full impact sample 


Atlanta 


2,936 


9,064 


9,236 


-172 


0.36 


Grand Rapids 


3,099 


10,414 


11,199 


-785 *** 


0.00 


Riverside 


4,980 


15,584 


n/a 


n/a 


n/a 


No high school diploma or GED 


Atlanta 


1,190 


10,209 


10,740 


-531 * 


0.07 


Grand Rapids 


1,268 


12,009 


12,914 


-905 ** 


0.03 


Riverside 


3,182 


17,171 


17,176 


-6 


0.99 


High school diploma or GED 


Atlanta 


1,742 


8,278 


8,252 


27 


0.91 


Grand Rapids 


1,827 


9,297 


10,024 


-727 ** 


0.03 


Riverside 


1,798 


13,504 


n/a 


n/a 


n/a 




Total months of welfare receipt in years 1 to 5 


Full impact sample 


Atlanta 


2,936 


34.4 


35.3 


-0.9 


0.17 


Grand Rapids 


3,099 


26.9 


28.2 


-1.4 ♦♦ 


0.02 


Riverside 


4,980 


27.8 


n/a 


n/a 


n/a 


No high school diploma or GED 


Atlanta 


1,190 


38.0 


40.5 


-2.5 ** 


0.02 


Grand Rapids 


1,268 


30.5 


32.1 


-1.5 


0.11 


Riverside 


3,182 


30.1 


30.0 


0.1 


0.92 


High school diploma or GED 


Atlanta 


1,742 


31.9 


31.9 


0.0 


0.97 


Grand Rapids 


1,827 


24.3 


25.6 


-1.3 * 


0.10 


Riverside 


1,798 


24.7 


n/a 


n/a 


n/a 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1. 
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Chapter 6 



Impacts on Income and Self-Sufficiency 



This chapter considers whether employment- and education-focused programs helped 
program group members reach a higher level of self-sufficiency and economic security than con- 
trol group members. First, the chapter analyzes whether programs led to impacts in. combined 
income over five years from earnings (net of payroll taxes), Earned Income Tax Credits (EITC), 
welfare, and Food Stamps. Next, the chapter considers whether program group members were 
more likely to leave welfare for employment than control group members. Impacts on these 
measures were calculated with administrative data. Finally, the chapter looks at program effects 
on receipt of income from earnings and other sources by sample members and other members of 
their household at the end of year 5, based on responses to the Five-Year Client Survey. 

Promoting self-sufficiency is an important goal for welfare-to-work programs, particularly 
in the current welfare environment. Programs that increase employment and raise income reduce 
the likelihood that families will return to welfare and/or experience long-term joblessness and hard- 
ship — a possibility under time-limited welfare if families exhaust their eligibility for assistance. 

The findings in this chapter may also foreshadow results on other outcomes for families 
and children discussed in Chapters 9 through 12. Recent research has indicated that programs 
that increase employment and raise income are likely to benefit children, whereas programs that 
increase employment without increasing income typically do not. Moreover, programs that in- 
creased income resulted in less stress for parents and less domestic violence.' 

I. Key Findings 

• During years 1 to 5, program group members in all sites became more self- 
sufficient than control group members by spending less time on welfare and 
receiving more of their total income in the form of earnings. 

• Most programs did not raise program group members’ income above control 
group levels, and some programs decreased income relative to the control 
group. Effects on income varied by site rather than program approach. 

• With some exceptions, programs led to more favorable effects (larger in- 
creases or smaller decreases) for sample members who had received a high 
school diploma or GED certificate before random assignment than for those 
who had not. 

• At the end of year 5, virtually all sample members reported having some 
source of income, either of their own or from other household members. How- 
ever, programs had little effect on income sources other than sample mem- 
bers’ earnings and public assistance. 



'Morris et al., 2001; Bos et al., 1999; Knox, Miller, and Gennetian, 2000. 
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II. Analysis Issues 



A. Defining Self-Sufficiency 

All welfare-to-work programs seek to increase self-sufficiency, although this concept may 
take on different meanings. Attaining self-sufficiency can mean ending welfare dependency, the 
primary goal of many welfare-to-work programs. As discussed in Chapter 5, all programs in the 
evaluation achieved this goal to some extent, although reductions in welfare receipt were small 
for some programs. Likewise, a program may promote self-sufficiency by increasing welfare re- 
cipients’ reliance on earnings rather than welfare benefits. This goal may be accomplished by en- 
couraging welfare recipients to combine work and welfare (at least in the short term) or by help- 
ing them to leave welfare for employment. As discussed in Chapter 4, most programs resulted in 
higher earnings, implying that they achieved this goal as well. 

Single mothers must obtain adequate resources to be truly self-sufficient, however. There- 
fore, a more comprehensive measure of self-sufficiency is income. In the context of an experi- 
mental evaluation, a program successful at improving self-sufficiency would reduce welfare use 
but leave program group members with more income than they would have had in the absence of 
the program. 

It should also be recognized that focusing solely on how programs affected sample mem- 
bers’ income might lead to inaccurate conclusions about whether sample members benefited 
from enrolling in a welfare-to-work program. As will be explored at the end of the chapter, in- 
come from a spouse, partner, or other friends and family members may have contributed signifi- 
cantly toward the well-being of sample members and their children. 

B. Direct and Indirect Effects on Self-Sufficiency and Income 

Welfare-to-work programs can directly increase self-sufficiency by increasing employ- 
ment and decreasing welfare receipt. They may also increase people’s income by helping them 
qualify for or find relatively well-paying jobs with benefits. The amount of effort that program 
staff devote to helping recipients who enter employment apply for the EITC, maintain eligibility 
for Food Stamps, and obtain medical coverage and child care assistance may also help partici- 
pants maintain or increase their income (or forgo costly expenditures for medical care and other 
necessities). Programs may not increase income, however, if they encourage people to take jobs 
that pay less than welfare, impose financial sanctions on those who have difficulty in finding 
work, or induce them to leave welfare without employment. 

Welfare-to-work programs can also indirectly affect self-sufficiency and income by 
changing welfare recipients’ social and family networks once they begin working. For example, 
welfare recipients who work for pay may be more likely to find spouses or partners who work, or 
may find job leads for other members of their household, thereby increasing the household’s in- 
come and economic security. Increases in employment and income may also improve people’s 
ability to purchase goods (like cars and clothing) and services (like reliable day care and health 
care) that support job retention and advancement and may help them obtain credit or save for the 
future. Alternatively, welfare-to-work programs that increase unstable employment or encourage 
recipients to leave welfare without employment may lead to immediate hardship and decrease 
people’s ability to maintain the social and material supports needed to find employment in the 



O 

ERIC 



-125- 



170 



future. Later chapters in this report discuss the effects of the programs on marriage, household 
composition, and material hardship. 

C. Measurement Issues 

This chapter estimates effects on income and self-sufficiency using administrative records 
and data from the Five-Year Client Survey. As discussed in Chapter 2, each of these data sources 
has its advantages and limitations. Measures of employment and welfare status and income are 
calculated from statewide unemployment insurance (UI) earnings records and welfare and Food 
Stamp payment records for all sample members. Estimates based on these administrative records 
are useful because they include everyone, cover the entire follow-up period, and likely include 
the primary sources of income received by most sample members.^ However, these estimates 
leave out other potentially important sources of income, including earnings not reported to the UI 
system, child support, other types of transfer payments, income from other household members 
(from earnings, public assistance, or other sources), or income from family and friends who live 
outside the household. Therefore, on the basis of administrative records alone some sample 
members may be incorrectly classified as not employed or as having no income. The Five-Year 
Client Survey includes these other sources of earnings and income, but for a much smaller group 
of sample members and for only one month at the end of year 5. Further, like all survey-based 
data, reported earnings and income are subject to recall error, nonreporting, and exaggeration. 

Finally, it should be kept in mind that this analysis does not consider program effects on 
work-related expenses (such as out-of-pocket expenses for child care and transportation), which 
decrease the income of sample members and their households.^ 



III. Impacts on Income 

A. Combined Income Over Five Years 

The upper panel of Table 6.1 shows average income over five years for program and con- 
trol group members in each program, as well as the impact of each program, measured as usual 
as the difference between the program and control groups. For this analysis, sample members’ 
income is the sum of their earnings, welfare payments. Food Stamps, and estimated Earned In- 
come Tax Credits (EITCs) less estimated payroll taxes.'* 



See, for example, Appendix Table E.4, which displays income receipt at the end of year 5, based on survey 
data. A relatively small percentage of respondents reported receipt of income from sources other than earnings, wel- 
fare,^and Food Stamps. The main exceptions are child support payments in Grand Rapids and Portland. 

See Chapter 10 for a discussion of out-of-pocket expenses for child care. 

It was beyond the scope of this evaluation to measure the EITC and payroll taxes directly. Instead, these out- 
comes were estimated on the basis of sample members’ measured earnings, mles for calculating the EITC and taxes, 
and assumptions about the percentage of sample members who applied for the EITC on their federal income tax re- 
turn. For each sample member, EITC calculations for years 1 to 5 use the tax mles for the calendar years in which 
the quarters 4, 8, 12, 16, and 20 occurred. Tax years range from 1992 through 1996 to 1995 through 1999, depend- 
ing on when sample members were randomly assigned. Calculations assume an 80 percent take-up rate (see Scholz, 
1996). Specifically, sample members’ EITCs were estimated for each year of follow-up and then multiplied by 0.8^ 
Estimated payroll taxes were calculated by multiplying earnings by 7.65 percent, the tax rate during most of the fol- 
low-up period. 
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National Evaluation of Welfare-to-Work Strategies 
Table 6.1 

Impacts on Combined Income in Years 1 to 5 



Site. and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Percentage 
Change (%) 




Average combined income in years 1 to 5 tS) 


Atlanta Labor Force Attachment 


2,938 


41,138 


39,987 


1,152 


2.9 


Atlanta Human Capital Development 


2,992 


41,120 


39,987 


1,133 


2.8 


Grand Rapids Labor Force Attachment 


3,012 


40,739 


42,172 


-1,433 * 


-3.4 


Grand Rapids Human Capital Development 


2,997 


40,925 


42,172 


-1,247 


-3.0 


Riverside Labor Force Attachment 


6,726 


38,929 


39,804 


-875 


-2.2 


Lacked high school diploma or basic skills 


3,125 


37,030 


38,311, 


-1,280 


-3.3 


Riverside Human Capital Development 


3,135 


35,924 


38,311 


-2,387- *** 


-6.2 


Columbus Integrated 


4,672 


44,037 


44,478 


-441 


-1.0 


Columbus Traditional 


4,729 


44,232 


44,478 


-246 


-0.6 


Detroit 


4,459 


48,256 


47,685 


571 


1.2 


Oklahoma City 


8,677 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


43,677 


41,807 


1,870 


4.5 




Earnings as a percentage of combined income (%) 


Atlanta Labor Force Attachment 


2,938 


36.0 


32.4 


3.5 *** 


10.9 


Atlanta Human Capital Development 


2,992 


35.4 


32.4 


3.0 *** 


9.1 


Grand Rapids Labor Force Attachment 


3,012 


43.7 


39.8 


3.9 *** 


9.9 


Grand Rapids Human Capital Development 


2,997 


42.0 


39.8 


2.3 ** 


5.7 


Riverside Labor Force Attachment 


6,726 


34.1 


28.6 


5.5 *** 


19.1 


Lacked high school diploma or basic skills 


3,125 


28.5 


22.8 


5.7 *** 


25.0 


Riverside Human Capital Development 


3,135 


26.3 


22.8 


3.5 *** 


15.2 


Columbus Integrated 


4,672 


48.6 


44.3 


4.3 *** 


9.8 


Columbus Traditional 


4,729 


46.9 


44.3 


2.6 *** 


5.8 


Detroit 


4,459 


35.8 


33.1 


2.7 *** 


8.0 


Oklahoma City 


8,677 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


45.5 


38.7 


6.8 *** 


17.7 



SOURCE; MDRC calculations from state and county administrative records. 
NOTES; See Appendix A. 1 . 
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Over five years, control group members in five sites received between $40,000 and $45,000 
in combined income, or about $8,000 to $9,000 per year. Control group members in Detroit re- 
ceived a slightly larger amount: $9,500 per year (Table 6.1, upper panel).^ Control group members 
differed in the proportion of their combined income that was made up by earnings: about 30 percent 
of combined income in Atlanta, Detroit, and Riverside over five years, compared with about 40 
percent in Columbus, Grand Rapids, and Portland (Table 6.1, lower panel). 

Programs in the NEWWS Evaluation generally did not increase sample members’ income 
relative to the control group. More often, they helped program group members substitute earn- 
ings and EITC for welfare and Food Stamps but left them with about the same level of income as 
control group members. This finding applies to LFA and HCD programs and, more generally, to 
employment- and education-focused programs. 

There were some differences in program effects on combined income, but impacts varied 
by site rather than program approach. In Atlanta LFA and HCD and Portland, program group 
members received from 2.8 to 4.5 percent more in combined income than control group mem- 
bers, although these program-control group differences were just above the 0.10 level of statisti- 
cal significance.^ (See Table 6.1.) In contrast, both programs in Grand Rapids and Riverside re- 
duced combined income over five years by 2.2 to 6.2 percent below control group levels.’ The 
programs in Columbus and Detroit affected income very little. 

The general inability of welfare-to-work services and mandates to increase income is not 
specific to programs studied in the NEWWS Evaluation. Results from San Diego’s Saturation 
Work Initiative Model (SWIM), California’s Greater Avenues for Independence (GAIN) program, 
Los Angeles County s Jobs First GAIN program, and Florida’s Project Independence were similar: 
even when programs increased earnings, they seldom increased income very much.^ 

B. Combined Income by Year 

Figure 6.1 and Appendix Table E.l display the combined income of program and control 
group members during each year of follow-up. Year-by-year trends in income varied considera- 
bly by site. In Grand Rapids, Detroit, and Atlanta, control group members’ average combined 
income increased by about $900 to nearly $1,800 between years 1 and 5. In contrast, control 
group members’ combined income decreased during these years in Portland and Riverside by 



^As noted above, it was assumed that 80 percent of sample members with earnings received EITCs. To test the 
effects of this assumption, sample members’ combined income was recalculated, assuming that 100 percent of sam- 
ple members with earnings received the EITC. The higher rate made the intact estimates on combined income more 
positive by about $170 for Portland and $90 for Riverside LFA. However, impact estimates for other programs 
changed very little. (Results not shown.) 

®P-values were 0. 1 1 for Atlanta LFA and HCD and 0. 1 5 for Portland. 

The decreases for Grand Rapids LFA (-$1,433) and Riverside HCD (-$2,387) were statistically significant. The 
decreases for Grand Rapids HCD (-$1,247, p-value = 0.11) and Riverside LFA (-$875, p-value = 0.16) were just 
above the 0. 1 level of statistical significance. 

*See Riccio, Friedlander, and Freedman, 1994, for GAIN; Hamilton and Friedlander, 1989, for SWIM; Freed- 
man et al., 2000b, for Jobs-First GAIN; and Kemple, Friedlander, and Fellerath, 1995, for Project Independence. See 
Bloom and Michalopoulos, 2001, for a synthesis of these and other studies. 
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National Evaluation of Welfare-to-Work Strategies 
Figure 6.1 

Impacts on Combined Income in Years 1 to 5 
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(continued) 
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Figure 6.1 (continued) 
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Figure 6.1 (continued) 
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SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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about $700 and $1,050, respectively. Control group members in Columbus received about the 
same amount of income during years 1 and 5. Trends for program group members were similar.^ 

Programs may have affected income in particular years, but these impacts can be ob- 
scured when impacts are calculated for the entire five-year follow-up. For example, it would be 
important to know whether programs increased income during the first three years of follow-up 
when no control group members had access to program services. Likewise, increases or decreases 
in income during year 5 would suggest future trends.'® 

According to Figure 6.1 and Appendix Table E.l, program impacts on combined income 
were concentrated during certain follow-up years. For example, during the first three years after 
random assignment, the Atlanta LFA program led to significantly greater income on average for 
program group members than for control group members. Similarly, the Atlanta HCD program 
achieved statistically significant increases in combined income in years 2 to 4. However, neither 
Atlanta program significantly increased income in year 5 of the follow-up period. Portland also 
increased program group members’ income above control group levels in years 2 to 5 — by $400 
to $600 per program group member." These three programs were not the norm, however. Grand 
Rapids LFA and HCD were more typical. The reductions in income stemming from the two 
Grand Rapids programs were spread out over the entire five-year period, even though the differ- 
ences were statistically significant only in years 1 and 2 for Grand Rapids LFA and years 1 and 4 
for Grand Rapids HCD. 

Perhaps the most troubling results were in the Riverside HCD program, which left pro- 
gram group members with significantly less income than control group members in each year of 
the follow-up period. Reductions in income were especially large in years 2 to 4, when the aver- 
age program group member had between $460 and $560 less in combined income than the aver- 
age control group member. 

C. Impacts on Combined Income for Educational Attainment Subgroups 

Figure 6.2 shows program effects on combined income over five years for high school 
graduates and nongraduates. It appears that income was more adversely affected for high school 
nongraduates. Of the nine programs for which results can be compared for the two groups, six 
either increased income more for high school graduates than nongraduates or reduced income 
less for graduates than for nongraduates. The differences are most striking in Atlanta, where the 
average high school graduate program group members had about $1,500 to $2,000 more income 



’Trends in average combined income for the program or control group may be misleading because they do not 
include other sources of income for sample members and their households. However, it is reasonable to assume that 
excluding these sources did not bias the cumulative or yearly estimates of impacts on combined income. 

Appendix Table E.2 shows the cumulative effects of the programs on income from earnings and public assis- 
tance (that is, without estimated EITC payments and payroll taxes) over the first three years and for the last quarter 
of year 5. 

For Portland, only the impact for year 4 was statistically significant, although p-values for years 2 (.139) and 3 
(.102) were just above the .1 level of statistical significance. The increase in year 5 of $470 had a p-value of .245. 
Detroit s program also led to higher income during years 2 to 4, but the differences were small and not statistically 
significant. 
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National Evaluation of Welfare-to-Work Strategies 
Figure 6.2 

Impacts on Combined Income in Years 1 to 5, 
by High School Diploma or GED Status 
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than comparable control group members, but average nongraduate program group members had 
about the same income as comparable control group members. The main exception was Portland, 
which increased combined income for nongraduates by nearly $2,700 (or 7 percent) above the 
control group mean, but led to a much smaller gain for graduates. However, the program-control 
group difference for nongraduates was not statistically significant (p-value = .18). 

There is not much indication that program approach produced important differences in 
income gains or losses. Both Atlanta programs had virtually no effect on income for high school 
non^aduates, and both had much larger effects for high school graduates. Likewise, both Grand 
Rapids programs resulted in lower combined income for program group members than control 
group members in each subgroup. On the other hand, the income losses from the Grand Rapids 
LFA program were more concentrated among high school graduates, whereas the income losses 
from the Grand Rapids HCD program were more concentrated among high school nongraduates. 



IV. Impacts on Employment and Welfare Status Over Five Years 

Income is perhaps the most comprehensive measure of economic well-being, but wel- 
fare-to-work programs can also help welfare recipients attain greater self-sufficiency by help- 
ing them find jobs and leave public assistance. Table 6.2 shows the degree to which the 
NEWWS programs succeeded in this regard by showing outcomes and impacts for four com- 
posite measures; (1) the proportion of people who were working and not receiving welfare, (2) 
the proportion who combined work and welfare, (3) the proportion who were on welfare and 
not working, and (4) the proportion who were neither working nor receiving welfare.'^ The ta- 
ble reports the percentage of the five-year (or 20-quarter) follow-up that people spent on aver- 
age in each employment-welfare status. 

Most programs led to small increases (of less than 5 percentage points) in the likelihood 
that people would work without receiving welfare (Table 6.2, first panel). The Portland program 
produced the largest impact, an increase of 7.3 percentage points above the control group level. 
Except in Riverside, programs did not affect the likelihood that sample members would combine 
work and welfare (Table 6.2, second panel). The increase for the Riverside programs probably 
reflects their overall gains in employment and the relatively high grant levels and generous earn- 
ings disregards in California.'^ 



Appendix Table E.3 shows similar results for the last quarter of year 5. Since the programs in general had 
small effects on employment during the last quarter of year 5, they also had relatively small effects on these out- 
comes. However, several of the programs continued to significantly reduce the proportion of people who were on 
welfare and not working. Notably, Portland increased the proportion of sample members who were working and not 
receiving welfare by 5.8 percentage points relative to the control group, by far the largest effect on this measure of 
self-sufficiency. 

' The other three employment-focused programs led to 1 percentage point gains in this measure. The fact that 
people were not able to leave welfare when they went to work implies that they earned fairly little, perhaps because 
they worked part time or earned close to the minimum wage. In addition, sample members may have received wel- 
fare and earnings at different times during a quarter as they transitioned from welfare to work or from work back to 
welfare. MDRC did not investigate these issues. 



- 134 - 



179 



National Evaluation of Welfare-to-Work Strategies 
Table 6.2 

Impacts on Percentage of Five-Year Follow-Up That People 
Spent in Each Employment-Welfare Status 

Sample Program Control Difference Percentage 
Site and Program Size Group (%) Group (%) (Impact) Change (%) 



Atlanta Labor Force Attachment 2,938 

Atlanta Human Capital Development 2,992 

Grand Rapids Labor Force Attachment 3,012 

Grand Rapids Human Capital Development 2,997 

Riverside Labor Force Attachment 6,726 

Lacked high school diploma or basic skills 3,125 

Riverside Human Capital Development 3,135 

Columbus Integrated 4,672 

Columbus Traditional 4,729 

Detroit 4,459 

Oklahoma City 8,677 

Portland 4,028 

Atlanta Labor Force Attachment 2,938 

Atlanta Human Capital Development 2,992 

Grand Rapids Labor Force Attachment 3,012 

Grand Rapids Human Capital Development 2,997 

Riverside Labor Force Attachment 6,726 

Lacked high school diploma or basic skills 3,125 

Riverside Human Capital Development 3,135 

Columbus Integrated 4,672 

Columbus Traditional 4,729 

Detroit 4,459 

Oklahoma City 8,677 

Portland 4,028 



Employed and not on welfare 



24.2 


21.4 


2.8 *** 


13.1 


23.8 


21.4 


2.4 ** 


11.2 


29.4 


27.1 


2.3 ** 


8.6 


28.6 


27.1 


1.5 


5.4 


19.5 


17.3 


2.2 *** 


12.9 


15.8 


13.8 


2.1 *** 


15.2 


14.8 


13.8 


1.1 


7.8 


35.1 


30.3 


4.8 *** 


15.8 


33.2 


30.3 


2.9 *** 


9.4 


21.6 


19.7 


1.9 *** 


9.5 


n/a 


n/a 


n/a 


n/a 


35.1 


27.8 


7.3 *** 


26.4 



Employed and on welfare 



18.5 


17.5 


1.0 


5.5 


17.8 


17.5 


0.3 


1.7 


19.7 


18.4 


1.3 ** 


7.1 


18.7 


18.4 


0.3 


1.6 


14.3 


10.8 


3.5 *** 


32.2 


14.0 


9.7 


4.2 


43.5 


12.6 


9.7 


2.8 *** 


29.0 


16.1 


18.7 


-2.7 *** 


-14.2 


17.4 


18.7 


-1.3 *** 


-7.2 


19.8 


20.5 


-0.7 


-3.2 


n/a 


n/a 


n/a 


n/a 



12.1 



11.2 



0.9 



7.9 



(continued) 
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Table 6.2 (continued) 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 






Not emoloved and on welfare 




Atlanta Labor Force Attachment 


2,938 


42.0 


47.4 


-5.4 ♦♦♦ 


-11.4 


Atlanta Human Capital Development 


2,992 


44.1 


47.4 


-3 3 *** 


-6.9 


Grand Rapids Labor Force Attachment 


3,012 


29.4 


37.2 


-7.8 ♦♦♦ 


-21.0 


Grand Rapids Human Capital Development 


2,997 


32.4 


37.2 


-4.8 ♦♦♦ 


-12.9 


Riverside Labor Force Attachment 


6,726 


35.0 


43.7 


-8.6 ♦♦♦ 


-19.8 


Lacked high school diploma or basic skills 


3,125 


39.3 


48.5 




-19.1 


Riverside Human Capital Development 


3,135 


40.4 


48.5 


-8.1 ♦♦♦ 


-16.7 


Columbus Integrated 


4,672 


26.4 


29.7 


-3 3 *** 


-11.1 


Columbus Traditional 


4,729 


27.3 


29.7 


_2 4 ♦♦♦ 


-8.2 


Detroit 


4,459 


43.4 


45.5 


-2.1 ♦♦ 


-4.5 


Oklahoma City 


n/a 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


24.9 


34.8 


_9 9 ♦♦♦ 


-28.4 






Not emnloved and not on welfare 




Atlanta Labor Force Attachment 


2,938 


15.4 


13.7 


1.6 ♦ 


12.0 


Atlanta Human Capital Development 


2,992 


14.3 


13.7 


0.6 


4.3 


Grand Rapids Labor Force Attachment 


3,012 


21.5 


17.3 


4.2 *** 


24.2 


Grand Rapids Human Capital Development 


2,997 


20.3 


17.3 


3 Q ♦♦♦ 


17.5 


Riverside Labor Force Attachment 


6,726 


31.2 


28.2 


2 9 ♦♦♦ 


10.4 


Lacked high school diploma or basic skills 


3,125 


30.9 


28.0 


2 9 ♦♦♦ 


10.5 


Riverside Human Capital Development 


3,135 


32.2 


28.0 


4.2 *** 


15.0 


Columbus Integrated 


4,672 


22.4 


21.2 


1.1 


5.4 


Columbus Traditional 


4,729 


22.1 


21.2 


0.9 


4.2 


Detroit 


4,459 


15.2 


14.4 


0.8 


5.9 


Oklahoma City 


n/a 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


27.9 


26.3 


1.6 


6.3 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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The largest change for most programs was in the likelihood that people would rely on wel- 
fare without working (Table 6.2, third panel). In general, employment-focused programs produced 
the largest reductions in the proportion of follow-up quarters not employed and on welfare. The 
Portland employment-focused program produced the largest impact among all programs — 9.9 per- 
centage points below the control group level. The three LFA programs led to reductions in this 
status of between 5.4 percentage points (Atlanta) and 8.6 percentage points (Riverside). Among 
education-focused programs, only Riverside HCD led to a comparable reduction. All other educa- 
tion-focused programs (including Atlanta and Grand Rapids HCD) decreased the proportion of fol- 
low-up quarters not employed and on welfare by less than 5 percentage points. 

Finally, five programs (the three LFA programs and Grand Rapids and Riverside HCD) 
led to small but statistically significant increases in the proportion of follow-up quarters not em- 
ployed and not on welfare. These programs probably encouraged some people to leave welfare 
for reasons other than employment. Notably, the LFA and HCD programs in Grand Rapids and 
Riverside also led to the largest reductions in combined income. 

The proportion of follow-up quarters that program group members spent neither working 
nor receiving welfare (Table 6.1, fourth panel) ranged fi'om about 15 percent in Atlanta and De- 
troit to more than 30 percent in Riverside. These program group members may have experienced 
severe financial hardship during the follow-up period. However, it is important to keep in mind 
the measure includes only income fi'om earnings and welfare. It does not include income fi'om 
Food Stamps''* or other forms of public assistance, income of other household members, or earn- 
ings fi'om jobs that were out of state or not reported to the state’s UI earnings system.'^ 

V. Comparing the Effects of the LFA and HCD Programs on Income 

So far, the chapter has made informal comparisons of the effects of employment- and 
education-focused programs on measures of income and self-sufficiency. This section summa- 
rizes the results of a more precise test of the relative effects of the LFA and HCD programs in 
Atlanta, Grand Rapids, and Riverside on combined income over five years. (See Table 6.3.) Not 
surprisingly, there was no statistically significant LFA-HCD difference in combined income for 
the full samples in Atlanta and Grand Rapids. As noted above, both programs in each site led to 
similar effects on income: an increase for Atlanta LFA and HCD relative to the control group and 
a reduction for Grand Rapids LFA and HCD (Table 6.1). 

The small overall difference in income generated by the two approaches masks larger dif- 
ferences for educational attainment subgroups. For high school nongraduates, the LFA program 



'“'Across all sites, program and control group members received income from Food Stamps only (no earnings or 
welfare payments) during 2 to 7 percent of follow-up quarters. Programs did not affect the likelihood of having in- 
come from Food Stamps only. 

'^Appendix H compares employment levels and impacts calculated with UI earnings and survey data. As dis- 
cussed in Appendix G, survey respondents in Riverside and Portland reported considerably more employment than 
was recorded on UI earnings records. Furthermore, as \vill be discussed in greater detail below, according to re- 
sponses to the Five-Year Client Survey, virtually all sample members had some type of income in the month prior to 
completing the survey. (See Appendix Table E.4.) 
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National Evaluation of Welfare-to-Work Strategies 
Table 6.3 

LFA-HCD Differences in Combined Income in Years 1 to 5 



Site and Program 


Sample 

Size 


LFA($) HCD($) 


Difference 

(Impact) 


p-value 


Full impact sample 


Atlanta 


2,936 


41,138 


41,120 


19 


0.98 


Grand Rapids 


3,099 


40,739 


40,925 


-186 


0.81 


Riverside 


4,980 


38,929 


n/a 


n/a 


n/a 


No high school diploma or GED 


Atlanta 


1,190 


37,490 


36,890 


599 


0.52 


Grand Rapids 


1,268 


37,015 


35,835 


1,180 


0.23 


Riverside 


3,182 


37,030 


35,924 


1,106 


0.17 


High school diploma or GED 


Atlanta 


1,742 


43,596 


43,994 


-397 


0.70 


Grand Rapids 


1,827 


43,249 


44,489 


-1,240 


0.26 


Riverside 


1,798 


41,433 


n/a 


n/a 


n/a 



SOURCE: MDRC calculations from state and coimty administrative records. 
NOTES: See Appendix A. 1 . 
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group had higher income than the HCD program group in each site. Although the difference was 
not statistically significant in any one site, a simple average of the impacts across the three sites in- 
dicates that the three LFA programs as a group resulted in nearly $1,000 more income over five 
years than the HCD programs. Moreover, this LFA-HCD difference for high school nongraduates 
was statistically significant. These results, along with the LFA programs’ larger effects on employ- 
ment and earnings, suggest that job search was a better approach for helping welfare recipients with 
low educational attainment than the type of mandatory adult basic education services provided by 
the HCD programs. It should be recalled, however, that both the LFA and HCD programs in Grand 
Rapids and Riverside decreased the income of nongraduates relative to the control group. 

Among high school graduates and GED recipients, on the other hand, HCDs in Atlanta 
and Grand Rapids received more income over five years than LFAs. In neither site, however, was 
the difference statistically significant; the same result occurred when LFA and HCD impacts 
were averaged across the two sites. 

VI. Impacts on Respondent and Household Income at the End of Year 5 

A. Additional Sources of Income for Respondents 

Survey respondents were asked whether they had received income from earnings, welfare. 
Food Stamps, child support payments. Supplemental Security Income (SSI) payments. Social Se- 
curity benefits, or “other income” at the end of year 5. (See Appendix Table E.4, middle panels.) 
The last four types of income were not recorded from administrative records and not included in 
the measures of combined income presented above. Under some circumstances, programs may 
affect levels of receipt of child support payments. For instance, program group members were 
likely to have had more contact with the welfare department than control group members and 
may have received more help in seeking a child support order. Similarly, programs may have in- 
creased SSI receipt if program group members with disabilities had greater contact with the wel- 
fare department than control group members.’^ 

It turns out, however, that programs had only small and scattered effects on receipt of 
these income sources at the end of year 5. Only Atlanta HCD increased receipt of child support 
payments — by 4.2 percentage points above the control group level of 13.7 percent. Two other 
programs. Grand Rapids HCD and Riverside LFA, produced a similar increase in SSI receipt. 
Finally, Riverside HCD reduced “other income” by 4.5 percentage points. 

B. Household Income 

As noted above, it is important to consider whether other household members received 
income from work or other sources and may have contributed toward the purchase of food, cloth- 
ing, shelter, and other necessities for respondents and their children and helped respondents to 

1 7 

experience a higher standard of living than they could have attained from their income alone. 



'^Some respondents also reported earnings that had not been recorded by the UI system. See Appendix H. 

‘^Some household members who received income may not have contributed financial support to sample mem- 
bers and their children. For instance, people receiving SSI or other types of assistance may have needed to use all of 
their income to meet their own needs. 
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Higher household income may also be associated with favorable outcomes for children, an issue 
explored in Chapters 1 1 and 12. 

Respondents to the Five-Year Client Survey reported whether other members of their 
household received income from employment, welfare. Food Stamps, child support, SSI, social 
security, or other sources at the end of year 5. (See Appendix Table E.4, lower panels.) About 55 
percent of control group respondents in Grand Rapids, Portland, and Riverside and about 40 per- 
cent in Atlanta reported living with someone who received income from one or more of these 
sources. Most programs did not affect the likelihood of living with others who received income, 
although the decrease for Riverside LFA (of 4.2 percentage points) was just above the 10 percent 

1 R 

level of statistical significance. 

A large majority of control group respondents reported that they were living in a house- 
hold in which at least one person worked for pay: from 65.5 percent in Atlanta to 80.3 percent in 
Grand Rapids (Appendix Table E.4, upper panels). Furthermore, a fairly large proportion of con- 
trol group members — 42 to 43 percent in Grand Rapids, Portland, and Riverside and 28 percent 
in Atlanta — lived with at least one person who was working for pay (Appendix Table E.4, lower 
panels). Fewer than half of these respondents (or from 10 to 19 percent of all control group 
members) reported that they were jobless but other household members were working for pay 
(Appendix Table E.5, lower panels). 

Programs did not affect the likelihood of living with another person who was the sole 
wage-earner in the household or living in a household in which the respondent and another 
household member were working for pay. Similarly, programs had no impact on the proportion 
of sample members who lived in a household in which monthly earnings for other household 
members totaled $1,000 or more (results not shown). 



'*A larger reduction for LFA nongraduates was statistically significant. The Portland program led to a program- 
control group difference of +6.0 percentage points that was not statistically significant (p-value = .21). The p-value 
for the Riverside LFA impact was .13. 
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Chapter 7 

What Works Best for Whom: Economic Effects by Subgroup 



Chapters 4-6 revealed the overall effects on economic outcomes of the NEWWS programs 
studied. With regard to earnings, for example, employment-focused programs had larger immediate 
effects than education-focused programs, Portland had large and persistent effects, and education- 
focused programs in Detroit and Oklahoma City had relatively small effects. This chapter investi- 
gates whether some groups were affected more or less than others. In particular, program effects on 
average earnings, welfare benefits, and income are compared for long-term and short-term welfare 
recipients; those who had worked in the year prior to random assignment and those who had not; by 
race and ethnicity; and for groups defined by whether they faced multiple barriers to work as long- 
term welfare recipients, high school dropouts, or long-term unemployed. 

Knowing how welfare-to-work services affect various subgroups can help in deciding 
where to target new resources or where to develop new services. For example, recipients who are 
more disadvantaged and who are likely to have the most difficulty finding a job will be particularly 
at risk of losing income if they lose eligibility for benefits under TANF. Programs that showed posi- 
tive effects for these recipients may serve as good models under time-limited welfare. 

Results for more job-ready recipients are also of interest. Programs may only have as- 
sisted these individuals to secure jobs more quickly than they would have otherwise, but have 
had little long-term effect. If that was the case, targeting them might have been an inefficient use 
of resources. On the other hand, the programs might have helped job-ready recipients find 
higher-quality jobs, which could very well have had substantial positive effects on their long- 
term earnings. The results for job-ready subgroups can inform this debate, which is likely to be- 
come more heated as states continue to try different strategies under TANF. 

I. Key Findings 

• The programs generally had larger effects on earnings for people who 
had not worked in the year prior to random assignment than for those 
who had worked, and larger effects on earnings for long-term recipients 
than for short-term recipients. Programs that increased earnings more also 
tended to reduce cash assistance more. As a result, effects on cash assistance 
also were generally larger for those who had not worked recently and for long- 
term recipients. 

• There were often substantial differences in effects on earnings by level of 
disadvantage, but those differences were not consistent across programs. 

Some programs had much larger effects for the most disadvantaged than for 
the moderately disadvantaged. Other programs had much smaller effects for 
the most disadvantaged. Site may be more important than self-sufficiency ap- 
proach. For example, both Grand Rapids programs had larger effects for the 
moderately disadvantaged, but both Atlanta programs had larger effects for the 
most disadvantaged. 
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• In general, program effects on earnings were larger for Hispanic and Af- 
rican-American sample members than for white sample members. An im- 
portant exception was the Portland program, which increased earnings of 
white sample members by more than $6,000 over five years but did not change 
earnings of African-American sample members. It is unclear what caused this 
large difference, but the result should be interpreted with a great deal of cau- 
tion since the Portland sample contains few African-American control group 
members. 

• Although the more disadvantaged groups had higher earnings because of 
the programs, they still earned very little. For example, sample members 
without recent work experience had only about half as much in earnings as 
those with recent work experience, and the most disadvantaged group earned 
about half as much as the moderately disadvantaged group. 

• The programs did not systematically change income (from earnings, 
public assistance, and projected EITC payments net of payroll taxes) 
for any subgroups. This reflects the fact that welfare recipients who went 
to work lost some of their benefits, whether they were in the program or 
control group, while sanctions for those who failed to comply with program 
mandates may have reduced public assistance benefits by more than earnings 
for some people. 

• People in the LFA programs generally had higher earnings than those in 
the HCD programs. However, differences between the two approaches were 
generally small for most subgroups. 

• The Portland program, which was employment-focused but used a mix of 
job search and Adult Basic Education, had the largest or close to the 
largest effects on earnings for most subgroups. This suggests that the Port- 
land approach was effective for a wide range of people. 

II. Analysis Issues 

Subgroups were organized around three barriers to employment: high school education 
(high school graduates compared with nongraduates), recent work experience (those who had 
worked in the year prior to random assignment compared with those who had not), and welfare 
history (those who had ever been on welfare at least two years prior to random assignment com- 
pared with those who had not). Results by high school credential were presented in Chapters 4-6. 
This chapter presents results for the other barriers. In addition, the three individual barriers to 
employment were used to define three mutually exclusive subgroups based on relative disadvan- 
tage. The “most disadvantaged” were sample members who did not have a high school diploma 
or GED prior to random assignment, did not work in the year prior to random assignment, and 
were on welfare for two years or more prior to random assignment. “Moderately disadvantaged” 
sample members faced only one or two of the three barriers, while the “least disadvantaged” 
faced none. Finally, results are presented in this chapter for racial and ethnic groups. 
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Subgroups were identified using infomation collected just before individuals were r^- 
domly assigned. Because these groups were defined by pre-existing characteristics observed at 
study enrollment, control and program group members in the subgroup should be comparable at 
the time of random assignment, and any systematic differences that emerge between the two 
groups can be reliably attributed to the programs being studied. 

The chapter presents results for three outcomes: (1) earnings, (2) cash assistance, and (3) 
combined income from earnings, cash assistance. Food Stamps, and the federal Earned Income 
Credit (EITC) net of payroll taxes. The three outcomes represent three different perspectives. 
Many policymakers want to encourage welfare recipients to work; for them, the “best” program 
may be the one that increases earnings the most. Other policymakers may be primarily interested 
in reducing spending on welfare; for them the best program may be the one that reduces cash as- 
sistance the most. Welfare recipients and policymakers concerned about child and family poverty 
may care most about their total income; for them, the best program may be the one that increases 
income the most. 

For each outcome, the chapter focuses on cumulative dollar amounts over a five-year fol- 
low-up period. Although use of program services by control group members might have reduced 
the effects of some programs in years 4 and 5, five years of follow-up are used for two reasons. 
First, the program effects over five years are generally similar to their effects over three years. 
Second, an earlier report presented a detailed analysis of subgroup impacts over three years.' 

In analyzing subgroups, several types of comparisons are made, with each comparison an- 
swering a different question. The first question is whether there is evidence that the welfare-to- 
work programs taken as a whole — without regard to the approach they used or where they oper- 
ated — tended to affect a particular subgroup. For example, with welfare time limits, administra- 
tors probably want to make sure that long-term welfare recipients are able to find work and leave 
welfare; if the programs were not particularly effective at benefiting long-term recipients, poli- 
cymakers might want to target them for more resources or devise different and better services. 

A second question is whether the programs tended to have larger effects for one subgroup 
than another. Again, the answer to this question can help policymakers think about how to use their 
precious resources or whether new services should be developed. Suppose that long-term recipients 
were generally affected by the programs being studied. Suppose, however, that they were affected 
less than short-term recipients. This might suggest that more effort or different services should be 
considered for long-term recipients to increase the effectiveness of welfare-to-work services. 

A third question is whether one program approach benefited a subgroup, more than an- 
other approach or whether it benefited one subgroup more than another subgroup. Because there 
are fewer programs of each type, however, statements about the effects of particular program 
models might be more speculative. For example, Portland is the only NEWWS program that was 
employment-focused with varied first activities. Although its impacts on earnings were by far the 
largest — and this chapter shows that the effects were also the largest for most subgroups — it 
cannot be determined whether this is a consequence of Portland’s approach, the way sample 
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'Michalopoulos and Schwartz, 2001. 



members were chosen, the Portland economy (or other local factors), or unobserved differences 
between welfare recipients in Portland and in the other sites. 

Since the number of people in a subgroup is, obviously, less than the number in the full 
sample, it is consequently more difficult to confidently say that an individual program had an ef- 
fect for a subgroup than it is for the full sample, and it is more difficult to say whether the esti- 
mated effects are bigger for one group than another because of the program rather than by 
chance. However, the pattern of impacts across programs can provide statistical evidence that the 
programs taken as a whole had a particular effect, even if no individual program had a statisti- 
cally significant effect. For example, suppose the question is whether welfare-to-work programs 
have a larger impact on earnings for long-term welfare recipients or for short-term welfare re- 
cipients. If 9 or more programs had a larger impact for long-term recipients than for short-term 
recipients, say, the hypothesis that the impacts are the same for the two groups can be rejected at 
the 10 percent significance level — even if no single program had a statistically significant dif- 
ferent effect for long-term recipients than for short-term recipients. Likewise, if 10 or more pro- 
grams have an impact in the same direction, we can reject the hypothesis of no difference at the 5 
percent significance level, and if all 1 1 programs have an impact in the same direction, we can 
reject the hypothesis of no difference at the 1 percent significance level. ^ 

The most rigorous means of examining the effects of program models is to compare the 
effects of the three LFA programs with their HCD counterparts. The chapter consequently de- 
votes a section to this comparison. With only three programs of each type, however, it can be dif- 
ficult to draw firm conclusions about the relative benefits of the two approaches by subgroup. As 
mentioned above, it is harder to find statistically significant effects for a subgroup than for the 
full sarnple and unlikely that the impacts between two subgroups will be statistically significantly 
different in any specific site. Moreover, three sites are too few to use only the pattern of results to 
draw conclusions about the relative effectiveness of the two approaches unless the differences in 
impacts between the two approaches are large. If, for example, the LFA and HCD approaches are 
equally effective, then the chance that all three LFA programs would have larger impacts than all 
three HCD programs would be 12.5 percent, or greater than the usual threshold for drawing con- 
clusions based on statistical significance. However, it is extremely unlikely that all three LFA 
programs would increase earnings significantly more than all three HCD programs simply by 
chance, and statistically significant differences in all three sites would be enough to draw solid 
conclusions based on the statistical evidence. 



^The number of programs that produced a larger effect for one group than for another follows a binomial distri- 
bution. Significance levels were determined using this distribution, under the hypothesis that the programs had 
equally large effects for all subgroups. For example, if welfare-to-work programs do not affect the earnings of long- 
term welfare recipients, then the chance that all 1 1 programs would have had higher earnings for long-term recipient 
program group members than for long-term recipient control group members is 0.1 percent. Likewise, the chance 
that 10 or more of the programs would have had higher earnings for long-term recipient program group members 
than for long-term recipient control group members is 1.17 percent, and the chance that 9 or more of the programs 
would have had higher earnings for long-term recipient program group members than for long-term recipient control 
group members is 6.54 percent. 
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III. Impacts by Subgroup 

A. Welfare History 



The Family Support Act (FSA) of 1988, the provisions of which helped shape the pro- 
grams studied in this report, was designed to help individuals who were most likely to become 
long-term welfare recipients. The FSA required states to target this group for welfare-to-work 
resources and to offer services that were thought to provide them the greatest benefit. An impor- 
tant question about the NEWWS programs, therefore, is whether the programs had positive ef- 
fects for long-term recipients. 

Table 7.1 shows impacts on earnings; cash assistance payments; and income from earn- 
ings, cash assistance. Food Stamps, and the federal EITC net of payroll taxes, all measured over 
the five years following random assignment. Impacts are shown for each program for two sub- 
groups: long-term welfare recipients (those who had been on welfare for two years or more prior 
to random assignment) and short-term welfare recipients and welfare applicants (those who had 
been on welfare for less than two years prior to random assignment). To allow comparisons be- 
tween the two Riverside programs to be made, results for the Riverside LFA program are pre- 
sented both for the entire sample and for sample members considered in need of basic education 
(the only group assigned to the Riverside HCD program). 

, Long-term recipients. Table 7.1 shows that the NEWWS programs did increase earnings 
for long-term recipients. In all of the programs, long-term recipient program group members had 
higher earnings over three years than long-term recipient control group members. Impacts were 
highest in Portland, at nearly $6,000 over five years, with most programs having impacts between 
$1,000 and $3,700 per person. In all cases but three, moreover, the differences were statistically 
significant. 

In all 10 programs for which welfare benefits could be measured, long-term recipients in 
the program group received less in cash assistance than long-term recipients in the control group. 
In part, this is a natural consequence of going to work. In all sites, welfare recipients’ cash bene- 
fits were reduced somewhat when their earnings increased. In a number of programs, however, 
impacts on cash assistance were greater than impacts on earnings. In addition, as discussed in 
Chapter 5, sanctions would have resulted in welfare savings over and above the program effects 
on earnings, particularly in Grand Rapids and Columbus Integrated. Alternatively, some people 
may have left welfare because they were unable or unwilling to comply with program require- 
ments, and others may not have returned to welfare when they lost their jobs. 

Because the programs generally resulted in higher earnings but less cash assistance for 
long-term recipients than would have occurred otherwise, they generally had relatively small ef- 
fects on income from earnings, cash assistance. Food Stamps, and projected EITC payments net 
of payroll taxes. For programs outside Grand Rapids, the impacts on income were small enough 
that they could not reliably be attributed to the programs. Moreover, the estimated impact was 
negative in six programs and positive in four, providing a further indication that these programs 
did not systematically affect income. Nevertheless, the results in Grand Rapids may be consid- 
ered troubling. The reductions in income that were seen overall (in Chapter 6) appear to be con- 
centrated among long-term recipients. 
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National Evaluation of Welfare-to-Work Strategies 
Table 7.1 

Impacts on Selected Measures, by Welfare History 

Average Total Average Welfare Combined 

Earnings Payments Income 

Sample in Years in Years in Years 

Site and Program Size 1 to 5 ($) 1 to 5 ($) 1 to 5 ($) 



Atlanta Labor Force Attachment 2,063 

Atlanta Human Capital Development 2, 1 00 

Grand Rapids Labor Force Attachment 1 ,791 

Grand Rapids Human Capital Development 1 ,77 5 

Riverside Labor Force Attachment 3,510 

Lacked high school diploma or basic skills 1,831 

Riverside Human Capital Development 1 ,841 

Columbus Integrated 3,392 

Columbus T raditional 3,415 

Detroit 3,313 

Oklahoma City 2,076 

Portland 2,443 

Atlanta Labor Force Attachment 840 

Atlanta Human Capital Development 847 

Grand Rapids Labor Force Attachment 1,219 

Grand Rapids Human Capital Development 1 ,2 1 5 

Riverside Labor F orce Attachment 3,101 

Lacked high school diploma or basic skills 1,248 

Riverside Human Capital Development 1 ,238 

Columbus Integrated 806 

Columbus Traditional 793 

Detroit 1,015 

Oklahoma City 2,683 

Portland 1,494 



On welfare for two years or more 



2,522 ♦♦♦ 


-1 137 *** 


1,002 


2,059 ♦♦ 


-856 *** 


1,080 


1,221 


-2,931 *** 


-2,206 ♦♦ 


205 


-2,121 *** 


-2,341 ♦♦ 


3,657 ♦♦♦ 


-3,427 *** 


-568 


3,016 ♦♦♦ 


-3,302 *** 


-901 


2,582 ♦♦♦ 


-3,018 *** 


-1,177 


2,523 *** 


-1,686 *** 


-176 


1,424 ♦ 


-1,154 *** 


-129 


1,710 ♦ 


-860 *** 


370 


699 


n/a 


n/a 


5,859 *** 


-2,905 *** 


2,415 


On welfare for less than two years 




2,262 


-227 


1,623 


1,540 


-419 


850 


2,125 


-2,002 *** 


-280 


1,603 


-1,241 ♦♦♦ 


169 


1,506 


-1 979 *** 


-1,111 


1,550 


-2,565 


-1,624 


-415 


-2,855 ♦♦♦ 


-4,066 


-843 


-1,468 *** 


-3,362 


1,291 


-1,018 *** 


-917 


1,298 


-16 


1,300 


-236 


n/a 


n/a 


4,410 ♦ 


-2,638 ♦♦♦ 


1,538 



SOURCES: MDRC calculations from unemployment insurance (UI) earnings records and AFDC records. 

NOTES: In^acts on earnings were significantly different across subgroups in Riverside HCD. 

Impacts on AFDC were significantly different across subgroups in Atlanta LFA and Grand Rapids LFA. 
N/a = not applicable. 
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Short-term recipients. Table 7.1 indicates that the programs also generally had effects 
for short-term welfare recipients, although not as consistently as for long-term recipients. In all 
cases, short-term recipient program group members had lower cash assistance payments than 
their control group counterparts (though the difference was statistically significant in only 7 of 
the 10 programs). In only 8 of the 11 programs, however, did they have higher earnings — and 
the earnings impact was statistically significant for short-term recipients only in Portland. Par- 
ticularly troubling are the impacts for the Riverside HCD short-term recipients whose earnings 
were virtually unchanged, but whose cash assistance was reduced by more than $2,800 over five 
years. As a result of the program, short-term recipient program group members received more 
than $4,000 less in income from earnings, cash assistance, and Food Stamps than short-term re- 
cipient control group members. The Riverside HCD program was not alone: The Columbus Inte- 
grated program reduced income by more than $3,300 for program group members. 

Comparing short-term and long-term recipients. Should long-term recipients, short- 
term recipients, or both be targeted for new services? One means of addressing the question is to 
look at the relative effects of the programs on the two groups. Even though many of the programs 
were effective for both groups, long-term recipients were generally helped more than short-term 
recipients. In 9 of the 1 1 programs (the only exception was in Grand Rapids), the impact on earn- 
ings for long-term recipients was greater than the impact on earnings for short-term recipients. 
This suggests that the approach of the programs studied in this report was effective at increasing 
the earnings of long-term recipients. 

If a goal of welfare reform is to increase income, however, then these programs were gen- 
erally equally ineffective for long-term recipients and short-term recipients. If they are not al- 
ready doing so, states should consider further supplementing the earnings of welfare recipients or 
recent welfare recipients through enhanced earnings disregards, state Earned Income Tax Credits, 
or other means, to make it more likely that the programs that encourage them to work also help 
them to obtain greater financial resources.^ 

Another means of asking whether one group or another should be targeted for future ser- 
vices is to look at their outcomes. If the earnings levels of long-term recipients remained low — 
despite the fact that the programs were generally effective for this group — it might suggest the 
need for more or different services to further ameliorate their barriers to work. Across the 1 1 
programs there is generally a large gap in the earnings levels of the two groups: earnings of long- 
term recipients were generally about 60 to 75 percent of earnings for short-term recipients (result 
not shown in Table 7.1). For example, in the Atlanta LFA program, earnings for long-term re- 
cipients in the program groups were about $16,500 over five years compared with nearly $28,000 
for short-term recipients. 

B. Recent Work Experience 

Welfare-to-work programs are likely to have small effects if they offer services only to 
people whose barriers to work are so serious that they carmot take advantage of the services. 
People without much work experience could represent such a group; the fact that they have not 



^Michalopoulos and Berlin, 2001. 
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worked much may indicate the presence of barriers that are keeping them from working (if not 
the possibility that they would rather raise their children than work). Alternatively, welfare-to- 
work services could have small effects if control group members are so likely to work on their 
own (that is, without the benefit of program services) that services are unable to improve their 
outcomes. People who have worked recently clearly have the ability to find work and may repre- 
sent such a group. 

Recent work history does predict future earnings well and may help identify groups that 
could benefit from welfare-to-work services and groups that have less need to benefit. Across the 
1 1 programs people who had worked in the year prior to random assignment earned about twice 
as much as people who had not.“ The most extreme differences occurred in the Riverside HCD 
program, where control group members who had worked in the year prior to random assignment 
earned about $19,000 on average in the five years after random assignment, and control group 
members who had not worked earned less than $7,000. (In Portland, however, those who had 
worked recently earned only about 50 percent more than those who had not. This could reflect 
Portland’s very strong economy, the removal of the most disadvantaged participants from the 
study through up-front screening, or some other factor.) 

Table 7.2 addresses whether these differences in ability to earn without employment ser- 
vices translated into differential impacts for the two groups. The table offers evidence — but far 
from overwhelming evidence — that those who have not worked recently do benefit more from 
welfare-to-work services, at least in terms of increasing their earnings. 

Those without recent work experience. The programs generally led to higher earnings 
among those without recent work experience, which implies that the services helped those who 
had not worked recently find jobs (although there are many in this group who did not work 
much). In all 11 programs, earnings were higher for program group members who had not 
worked recently than for their control group counterparts. 

Once again, however, positive impacts on earnings did not generally translate into posi- 
tive impacts on income. The most positive findings were in Atlanta, where the programs resulted 
in income gains of more than $2,000 compared with what would have occurred otherwise. This 
might reflect Atlanta’s fill-the-gap budgeting, which allowed working welfare recipients to keep 
more of their welfare checks than they kept in other sites, or the relatively low welfare benefit 
levels in Georgia. 

Those with recent work experience. In all 11 programs, earnings were higher for pro- 
gram group members who had worked recently than for their control group counterparts. How- 
ever, the difference was statistically significant only in Detroit, and the impacts were typically 
not very big. Only in Detroit and Portland did the impact on earnings exceed $2,000. Despite the 
relatively small gains in earnings, the programs universally led to reduced welfare benefits for 
this group. In fact, the modest earnings gains combined with systematic reductions in welfare 
benefits led to the most systematic reductions in income for any subgroup: In 8 of the 10 pro- 
grams for which income could be measured, program group members with recent work experi- 



^Michalopoulos and Schwartz, 2001. 
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National Evaluation of Welfare-to-Work Strategies 
Table 7.2 



Impacts on Selected Measures, by Recent Work Experience 



Site and Program 


Average Total Average Welfare 
Earnings Payments 

Sample in Years in Years 

Size 1 to 5 ($) 1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 




Did not work in vear orior to random assienment 


Atlanta Labor Force Attachment 


1,869 


3,763 *** 


-928 *** 


2,323 *** 


Atlanta Human Capital Development 


1,937 


3,110 *** 


-780 *** 


2,020 ** 


Grand Rapids Labor Force Attachment 


1,527 


1,630 


-3,103 *** 


-2,157 ** 


Grand Rapids Human Capital Development 


1,489 


543 


-2 112 *** 


-1,896 * 


Riverside Labor Force Attachment 


4,010 


3,024 *** 


-3,027 *** 


-751 


Lacked high school diploma or basic skills 


2,074 


1,986 *** 


-3,286 *** 


-1,978 ** 


Riverside Human Capital Development 


2,065 


1,863 *** 


-2 969 *** 


-1,950 ** 


Columbus Integrated 


2,143 


2 925 *** 


-1,506 *** 


531 


Columbus Traditional 


2,160 


2,641 *** 


-1 225 *** 


938 


Detroit 


2,978 


904 


-400 


260 


Oklahoma City 


3,910 


245 


n/a 


n/a 


Portland 


2,317 


6,276 *** 


-3,620 *** 


2,035 




Worked in vear prior to random assignment 


Atlanta Labor Force Attachment 


1,069 


370 


-782 ** 


-691 


Atlanta Human Capital Development 


1,055 


238 


-666 ** 


-402 


Grand Rapids Labor Force Attachment 


1,485 


1,622 


-2,007 *** 


-586 


Grand Rapids Human Capital Development 


1,508 


1,240 


-1,478 *♦* 


-565 


Riverside Labor Force Attachment 


2,716 


1,788 


-2 222 *** 


-1,073 


Lacked high school diploma or basic skills 


1,051 


2,958 * 


-2,320 **♦ 


202 


Riverside Human Capital Development 


1,070 


403 


-2,723 


-2,962 * 


Columbus Integrated 


2,529 


1,374 


-1,533 **♦ 


-1,227 


Columbus Traditional 


2,569 


486 


-987 *** 


-1,160 


Detroit 


. 1,481 


2,592 * 


-907 ** 


1,155 


Oklahoma City 


4,767 


13 


n/a 


n/a 


Portland 


1,711 


3,522 


-1,620 *** 


1,408 



SOURCES: MDRC calculations from unemployment insurance (UI) earnings records and AFDC records. 

NOTES: Impacts on earnings were significantly different across subgroups in Atlanta LFA. 

Impacts on AFDC were significantly different across subgroups in Grand Rapids LFA and Portland. 
Impacts on total income were significantly different across subgroups in Atlanta LFA. 

N/a = not applicable. 
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ence ended up with lower income than their control group counterparts, though only in the River- 
side HCD program was this difference large enough to be statistically significant. 

Comparing those with and without recent work experience. The differences between 
those who had worked and those who had not are somewhat greater than the differences between 
long-term and short-term recipients. In the two Atlanta programs, for example, earnings impacts 
were virtually the same for long-term and short-term recipients, but were about $3,000 to $3,500 
greater for those without recent work experience than for those with recent work experience. In 
Portland, the impact on earnings for the group without recent work experience was nearly twice 
as large as the impact for those with recent work experience. In 9 of the 1 1 programs, in fact, im- 
pacts on earnings were larger for those who had not worked recently than for those who had. All 
of this suggests that people who have not worked in a while benefit the most (in terms of increas- 
ing their earnings and self-sufficiency, but not in terms of increasing their income) from these 
types of welfare-to-work programs. 

C. Composite Level of Disadvantage 

In summarizing results from several welfare-to-work programs from the 1980s, Fried- 
lander found that earnings gains were concentrated neither among groups of long-term recipients, 
who are expected to have a hard time finding work and leaving welfare, nor among groups such 
as new welfare applicants, who are most likely to work without assistance from a welfare-to- 
work program. Instead, Friedlander found the largest earnings gains among a middle group of 
welfare applicants who had spent some but not a great deal of prior time on welfare. In contrast, 
welfare savings came primarily from long-term recipients, especially those without a high school 
diploma or with little recent work experience.^ 

Do the NEWWS sites provide evidence that their approach changed these patterns by 
making the impacts for long-term recipients closer to those for short-term recipients? Results al- 
ready presented in this chapter differ from Friedlander’ s, in that impacts on earnings were gener- 
ally larger for the more disadvantaged group than for the less disadvantaged group. Table 7.3 
shows impacts for three groups defined by three barriers to work. (As noted above, the most dis- 
advantaged did not have a high school diploma or GED prior to random assignment, did not 
work in the year prior to random assignment, and were on welfare for two or more years prior to 
random assignment. The moderately disadvantaged faced only one or two of the three barriers, 
while the least disadvantaged faced none.) 

This method of defining the most and the least disadvantaged does a good job of finding 
groups that would fare well and poorly on their own, in terms of their earnings levels. In the five 
years after random assignment, average earnings for the most disadvantaged control group mem- 
bers ranged from less than $4,000 in Oklahoma City to about $11,500 in Detroit (not shown in 
Table 7.3). In contrast, earnings levels for the least disadvantaged control group members were at 
least three times higher in all sites, ranging from about $20,000 in Oklahoma City to nearly 
$45,000 in Columbus (and more than $40,000 in Detroit). 



^Friedlander, 1988. 



National Evaluation of Welfare-to-Work Strategies 
Table 7.3 



Impacts on Selected Measures, by Level of Disadvantage 



Site and Program 


Average Total Average Welfare 
Earnings Payments 

Sample in Years in Years 

Size 1 to 5 ($) 1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 






Most disadvanta&ed 




Atlanta Labor Force Attachment 


698 


1,946 * 


-766 ** 


1,285 


Atlanta Human Capital Development 


734 


-290 


-243 


-452 


Grand Rapids Labor Force Attachment 


458 


3,994 *** 


-4,028 *** 


-828 


Grand Rapids Human Capital Development 


453 


1,566 


-2,886 *** 


-2,136 


Riverside Labor Force Attachment 


1,362 


2,938 *** 


-3,592 *** 


-1,395 


Riverside Human Capital Development 


1,362 


3,279 *** 


-2,973 *** 


-445 


Columbus Integrated 


911 


2,206 * 


-2,047 *** 


-1,055 


Columbus Traditional 


901 


302 


-1,198 .*** 


-1,318 


Detroit 


1,119 


1,187 


-1,219 ** 


-827 


Oklahoma City 


429 


740 


n/a 


n/a 


Portland 


617 


3,162 


-2,862 ** 


491 






Moderatelv disadvantaged 




Atlanta Labor Force Attachment 


1,887 


3,406 *** 


-1,020 *** 


1,772 * 


Atlanta Human Capital Development 


1,911 


2,617 ** 


-904 *** 


1,489 


Grand Rapids Labor Force Attachment 


2,123 


270 


-2,396 *** 


-2,510 ** 


Grand Rapids Human Capital Development 


2,078 


934 


-1,749 *** 


-1,187 


Riverside Labor Force Attachment 


4,298 


3,035 *** 


-2,547 *** 


-204 


Riverside Human Capital Development 


3,049 


-1,307 


-2,185 *** 


-3,902 *** 


Columbus Integrated 


3,155 


2,663 ** 


-1,432 *** 


235 


Columbus Traditional 


3,236 


1,790 * 


-1,060 *** 


253 


Detroit 


3,018 


2,173 ** 


-271 


1,740 * 


Oklahoma City 


6,170 


255 


n/a 


n/a 


Portland 


2,803 


5,960 *** 


-3,318 *** 


1,935 
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Table 7.3 (continued) 



Site and Program 


Average Total 
Earnings 
Sample in Years 

Size 1 to 5 ($) 


Average Welfare 
Payments 
in Years 
1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 






Least disadvanta&ed 




Atlanta Labor Eorce Attachment 


353 


-1,121 


-119 


-1,347 


Atlanta Human Capital Development 


347 


709 


-514 


161 


Grand Rapids Labor Eorce Attachment 


431 


1,998 


-1,145 * 


665 


Grand Rapids Human Capital Development 


466 


1,869 


. -408 


1,664 


Riverside Labor Eorce Attachment 


1,066 


516 


-1,871 *** 


-2,162 


Columbus Integrated 


606 


-3,413 




-4,916 ♦ 


Columbus Traditional 


592 


-1,951 


-1,184 *** 


-4,376 


Detroit 


322 


-894 


-1,201 


-2,483 


Oklahoma City 


2,078 


-448 


n/a 


n/a 


Portland 


608 


2,430 


-159 


1,883 



SOURCES: MDRC calculations from unemployment insurance (UI) earnings records and AFDC records. 

NOTES: Impacts on earnings were significantly different across subgroups in Grand Rapids LEA, Riverside LEA, 
Riverside HCD, and Columbus Integrated. 

Impacts on AEDC benefits were significantly different across subgroups in Grand Rapids LEA and HCD and 
Portland. 

Impacts on total income were significantly different across subgroups in Riverside HCD, Columbus Traditiona 
Columbus Integrated, and Detroit. 

N/a = not applicable. 
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Examining subgroups by level of disadvantage may also help find more precisely defined 
groups who are helped most and least by the programs. This may help policymakers and welfare 
administrators decide whether the more disadvantaged or the less disadvantaged sample members 
benefit more from employment-focused or education-focused activities. It may also help them 
determine whether new services are needed for the more disadvantaged or the less disadvantaged 
sample members. In this sense, results by level of disadvantage are in the spirit of recent work on 
“profiling.”® 

Impacts for the most disadvantaged. These welfare-to-work programs generally in- 
creased earnings for the most disadvantaged, but their effects for this group were not as strong as 
for either long-term welfare recipients or those who had not worked in the year prior to random 
assignment. In 10 of the 11 programs, the most disadvantaged program group members earned 
more than their control group counterparts. Because there were relatively few people in this 
group, the impacts on earnings were statistically significant in only five of the programs. While 
several programs were moderately successful, increasing earnings by about $3,000 or more, just 
as many were not very effective, with impacts on earnings close to zero. 

Impacts on welfare benefits were more systematic, with significant reductions in 9 of the 
10 programs for which they could be measured. This same pattern was seen for the other sub- 
groups discussed above: more systematic changes in welfare payments than in earnings levels. 
As a result, the most disadvantaged program group members had lower income than their control 
group counterparts in 8 of the 10 programs (though the impact was not statistically significant in 
any of the programs). 

These results are somewhat different than the results for the two barriers described above, 
where earnings impacts were quite strong for those who had not worked in the year prior to ran- 
dom assignment and for long-term welfare recipients. This could imply that it is particularly dif- 
ficult to assist people with three barriers to work rather than possibly just one. On the other hand, 
it could indicate an important drawback to this method of counting barriers to work: Not all bar- 
riers are equal. In particular, Michalopoulos and Schwartz indicate that earnings gains were gen- 
erally larger for people with a high school credential than for those who lacked one.^ Thus, two 
of the barriers appear to be related to greater effects of the welfare-to-work programs, while a 
third barrier appears to be related to smaller effects of the programs. In such circumstances, un- 
derstanding the effects of multiple barriers requires more sophisticated methods than the catego- 
rization shown in Table 7.3. 

Comparing impacts by level of disadvantage. If Friedlander’s 1988 findings on pre- 
FSA programs hold for programs in the NEWWS Evaluation, then Table 7.3 should indicate that 
impacts are larger for the moderately disadvantaged than for the most disadvantaged. It does not. 
The impacts of the NEWWS programs on earnings were generally more broadly distributed than 
the impacts of the programs studied by Friedlander. Seven of the 1 1 programs had significant 
impacts on earnings for the moderately disadvantaged group, but 5 programs also had significant 
impacts on earnings for the most disadvantaged group. In addition, in both Grand Rapids pro- 



sper example, Eberts, 1997; and Rangarajan, Schochet, and Chu, 1998. 
^Michalopoulos and Schwartz, 2001. 
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grams and the Riverside HCD program earnings impacts were much larger for the most disad- 
vantaged than for the moderately disadvantaged, while in the two Atlanta programs and the Port- 
land program the opposite was true. 

In one way, however, results in Table 7.3 are similar to results from the pre-FSA pro- 
grams studied by Friedlander: The programs had little effect on earnings for the least disadvan- 
taged. Impacts on cumulative earnings were not statistically significant for any of the pro- 
grams. Moreover, just as many programs left the least disadvantaged program group members 
with higher earnings than their control group counterparts as left them with lower earnings. 
Despite the modest effects on earnings, each of the programs reduced cash assistance amounts 
four of them significantly so — although the programs did not have a consistently negative 
effect on income. 

D. Race and Ethnicity 

The final subgroups discussed in this chapter are defined by race and ethnicity. Results 
are presented for three groups: white, African-American, and Hispanic.® If minority sample 
members are faring much worse under these programs than white sample members, it might be a 
signal to policymakers and administrators that something is preventing minority sample members 
from fully participating in or benefiting from the programs. It may mean that the services offered 
were not enough to overcome additional barriers to employment often faced by minority group 
members, such as living far from available jobs, having language barriers, and being discrimi- 
nated against by employers. 

Results in Table 7.4 provide little reason to be concerned. In general, impacts on earnings 
were larger for Afncan-American and Hispanic sample members than for white sample mem- 
bers. In seven of the nine programs where comparisons between white and Afncan-American 
sample members could be made (Atlanta had too few white sample members to make reliable 
comparisons), impacts on earnings were larger for Afncan-American sample members. For ex- 
ample, the Grand Rapids LFA program increased earnings by nearly $2,400 over five years for 
Afncan-American sample members, but had virtually no effect on earnings for white sample 
members. In all programs except Portland and Detroit, in fact, the average impact on earnings for 
Afncan-American sample members exceeded the average impact for white sample members by 
more than $1,100 over five years. 

Despite these differences in the impacts on earnings, the impacts on welfare benefits were 
fairly similar for Afncan-American and white sample members. In the Grand Rapids LFA pro- 
gram, for example, the impact on welfare benefits was about $2,300 for white sample members 
and about $2,500 for Afncan-American sample members. As a result, both programs in River- 
side and Grand Rapids significantly reduced income for white sample members but not for Afn- 
can-American sample members. 

In the five programs with enough Hispanic sample members to allow impacts to be re- 
liably measured, program effects on earnings were generally larger for Hispanic sample mem- 



®A few sample members were not part of any of these three groups. Because only Oklahoma City and Riverside 
had more than a handful of these people, results for them are not shown in Table 7.4. 
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National Evaluation of Welfare-to-Work Strategies 

Table 7.4 



Impacts on Selected Measures, by Ethnicity 



Site and Program 


Average Total Average Welfare 
Earnings Payments 

Sample in Years in Years 

Size 1 to 5 ($) 1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 






White. non-Hisoanic 




Atlanta Labor Force Attachment 


n/a 


n/a 


n/a 


n/a 


Atlanta Human Capital Development 


n/a 


n/a 


n/a 


n/a 


Grand Rapids Labor Force Attachment 


1,470 


-275 


-2,302 *** 


-3,168 *** 


Grand Rapids Human Capital Development 


1,515 


-394 


-1,569 *** 


-2,467 ** 


Riverside Labor Force Attachment 


3,464 


1,565 * 


-2,350 *** 


-1,426 


Lacked high school diploma or basic skills 


1,245 


2,158 * 


-2,818 *** 


-1,148 


Riverside Human Capital Development 


1,208 


623 


-2,482.*** 


-2,430 * 


Columbus Integrated 


2,161 


1,762 


-1,489 *** 


-734 


Columbus Traditional 


2,204 


73 


-829 *** 


-1,104 


Detroit 


481 


2,794 


-1,129 


819 


Oklahoma City 


5,109 


-241 


n/a 


n/a 


Portland 


2,754 


6,343 *** 


-2,971 *** 


2,730 * 






Black. non-Hisoanic 




Atlanta Labor Force Attachment 


2,791 


2,385 *** 


-831 *** 


1,180 


Atlanta Human Capital Development 


2,838 


1,828 ** 


-718 *** 


998 


Grand Rapids Labor Force Attachment 


1,214 


2,367 ** 


-2,462 *** 


-162 


Grand Rapids Human Capital Development 


1,158 


999 


-1,472 *** 


-445 


Riverside Labor Force Attachment 


1,121 


3,775 ** 


-2,248 *** 


934 


Lacked high school diploma or basic skills 


501 


1,743 


-2,580 ** 


-1,233 


Riverside Human Capital Development 


510 


1,576 


-2,578 ** 


-1,614 


Columbus Integrated 


2,414 


2,528 ** 


-1,528 *** 


8 


Columbus Traditional 


2,431 


2,395 ** 


-1,265 *** 


390 


Detroit 


3,836 


1,375 


-557 ** 


500 


Oklahoma City 


2,484 


1,238 * 


n/a 


n/a 


Portland 


798 


-278 


-1,740 * 


-1,754 
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Table 7.4 (continued) 



Site and Program 


Average Total 
Earnings 
Sample in Years 

Size 1 to 5 ($) 


Average Welfare 
Payments 
in Years 
1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 








Hisoanic 




Atlanta Labor Force Attachment 


n/a 


n/a 




n/a 


n/a 


Atlanta Human Capital Development 


n/a 


n/a 




n/a 


n/a 


Grand Rapids Labor Force Attachment 


244 


4,973 


♦ 


-4,211 *** 


-464 


Grand Rapids Human Capital Development 


249 


6,270 




-4,002 *** 


999 


Riverside Labor Force Attachment 


1,858 


4,357 




-3,732 ♦♦♦ 


-420 


Lacked high school diploma or basic skills 


1,210 


3,639 




-3,669 ♦♦♦ 


-981 


Riverside Human Capital Development 


1,240 


3,018 




-3,536 ♦♦♦ 


-1,682 


Columbus Integrated 


n/a 


n/a 




n/a 


n/a 


Columbus Traditional 


n/a 


n/a 




n/a 


n/a 


Detroit 


n/a 


n/a 




n/a 


n/a 


Oklahoma City 


392 


-546 




n/a 


n/a 


Portland 


n/a 


n/a 




n/a 


n/a 



SOURCES: MDRC calculations from imemployment insurance (UI) earnings records and AFDC records. 

NOTES: Impacts on earnings were significantly different across subgroups in Grand Rapids LFA and HCD; Riverside 
LFA and HCD, and Portland. 

Impacts on AFDC benefits were significantly different across subgroups in Grand Rapids LFA and HCD. 

N/a = not applicable. 
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bers than for white or African-American sample members, although comparisons can be made 
across the three groups for only the four programs in Grand Rapids and Riverside because 
most of the sites had too few Hispanic sample members to provide reliable estimates of pro- 
gram impacts. The large earnings gains for Hispanic sample members were accompanied by 
large welfare reductions. 

One exception to the positive results for African-American sample members was in Port- 
land, where earnings increased by more than $6,000 for white sample members, but were little 
changed for African-American sample members. This result should be interpreted with a great 
deal of caution, however. The Portland sample used in this report contains only 101 African- 
American control group members, and the estimated effects for African-American sample mem- 
bers are consequently quite imprecise. In examining impacts through three years, Michalopoulos 
and Schwartz were able to use the entire Portland control group and found that the program in- 
creased earnings for African-American sample members by about $2,000 over three years.’ 

The reasons for the large discrepancy in the Portland effects are unclear. One possible ex- 
planation is that African-American control group members had higher rates of job search, em- 
ployment, and earnings than white control group members, which made it relatively harder to 
generate employment and earnings gains for African-Americans. According to the Five-Year Cli- 
ent Survey, about 47 percent of African-American control group members reported receiving 
some sort of job search assistance after entering the study compared with about 32 percent of 
white control group members. Perhaps as a result of their greater efforts to look for work, Afri- 
can-American control group members worked more often than white control group members (9.2 
quarters on average compared with 7.4 quarters) and they earned considerably more than white 
control group members (about $24,000 over five years compared with less than $20,000, shovm 
in Appendix Table F.4). However, it is unclear why Afiican-American control group members 
had such high rates of employment. 

Perhaps because of the high rates of participation in job search by African-American con- 
trol group members in Portland, the program effect on job search participation was somewhat 
lower for Afiican-Americans than for whites (a 22 percentage point increase for Afiican- 
Americans compared with a 30 percentage point increase for whites in the five years following 
random assignment). The smaller effect on job search assistance was offset somewhat by larger 
effects on any education and training for Afiican-American sample members (a 24 percentage 
point increase for Afiican-Americans compared with an 1 1 percentage point increase for whites). 
In addition, the program affected different types of education and training activities for the two 
groups, with the program’s effect on vocational training concentrated among Afiican-American 
sample members, its effect on basic education concentrated among white sample members, and 
its effect on post-secondary education about the same for white and Afiican-American sample 
members. It is important to note that the program increased both job search assistance and educa- 
tion and training for both white and Afiican-American sample members, so these results do not 
imply that lack of job search assistance or undue reliance on education were the cause of the low 
earnings gains for Afiican-Americans. 



’Michalopoulos and Schwartz, 2001. 
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IV. Comparing the LFA and HCD Programs 

In comparing the LFA and HCD programs in Atlanta, Grand Rapids, and Riverside, it is 
difficult to draw strong conclusions about the relative effectiveness of the job-search-first and 
education-first approaches over a five-year period. Across the 10 subgroups discussed in this 
chapter, only three impacts on earnings were numerically larger in an HCD program, of 27 com- 
parisons that were made. This suggests that the LFA programs were somewhat more effective 
than the HCD programs at increasing earnings. 

However, differences between the impacts of the two approaches were generally fairly 
small and generally not statistically significant. Moreover, for no subgroup was the evidence 
compelling. The largest differences between the two approaches were for the most disadvantaged 
sample members. In Atlanta, the earnings impact was more than $2,000 greater for the LFA pro- 
gram than for the HCD program ($1,946 compared with -$290; see Table 7.3). In Grand Rapids, 
the difference was nearly $2,500 ($3,994 compared with $1,566). However, in Riverside, the im- 
pact was slightly greater for the HCD program than for the LFA program. 

V. What Has Been Learned? 

Overall, the results suggest that these welfare-to-work programs, both education-focused 
and employment-focused, increased earnings and decreased welfare receipt for a wide range of 
subgroups. In particular, they suggest that the approach of the Family Support Act of 1988 was 
reasonably successful: The welfare-to-work programs did generally result in higher earnings for 
long-term welfare recipients. 

On the other hand, for the least disadvantaged (those sample members who have a high 
school diploma, have recent work experience, and have little prior welfare history) none of the 
programs increased earnings significantly, and the programs as a whole were just as likely to in- 
crease eaminp as to reduce them. This may reflect, again, the fact that the programs were forced 
by the provisions of the Family Support Act to concentrate resources on the most disadvantaged. 
Alternatively, it could be interpreted as evidence that concentrating services on the least disad- 
vantapd is an inefficient use of resources. Perhaps more advanced training, training that builds 
on skills already in place, is needed for the least disadvantaged. 

Impacts by subgroup bolster evidence from the full sample regarding the most successful 
programs and program approaches. LFA programs tended to have larger effects on earnings and 
welfare benefits for most subgroups, although differences were typically fairly small. The Port- 
land program was the most effective for the broadest range of subgroups. Of the seven subgroups 
compared (not counting those by race and ethnicity), Portland had the largest effect on earnings 
for six of the subgroups. This may reflect the program’s unique use of both job search and educa- 
tion as initial program activities, but it might also reflect the program’s use of job development, 
its experience running job search programs, its willingness to exempt welfare recipients who 
were perceived to be the hardest-to-employ from welfare-to-work services, or the interaction of 
these features with the city’s robust economy during this period. 
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Chapter 8 

Impacts on Health Care Coverage 

The programs in the NEWWS Evaluation were designed to move recipients from wel- 
fare to work, and the earlier chapters showed that many were successful in doing this. How- 
ever, moving from welfare to work leads to a loss in health coverage for some recipients if 
they cannot replace their lost Medicaid with private coverage. Many low-wage workers do not 
have employer-provided coverage, either because their employers do not offer it or they cannot 
afford the high premiums. 

This chapter examines how the NEWWS programs affected the health coverage of re- 
spondents and their children. Data on health coverage from the Five-Year Client Survey are 
available for 7 of the 1 1 programs. Health coverage status is an important aspect of family well- 
being, because it affects families’ access to care and the quality of care they receive. Children 
who are uninsured, for example, are much less likely to see doctors than their insured counter- 
parts, are less likely to have preventative care, and are more likely to have unmet health care 
needs.' Increasingly, families are without coverage: In 1997, for example, nearly half of all 
working poor parents were uninsured.^ 

I. Key Findings 

• The majority of control group respondents, ranging from 72 percent in Atlanta 
to 81 percent in Portland, had health coverage at the end of year 5. Most con- 
trol group members had coverage through public sources, such as Medicaid or 
other public programs, rather than employer-provided or other private sources. 
Children in the control group were somewhat more likely to have coverage 
than adults, owing to the greater number of public health programs available 
to low-income children. Coverage rates for children ranged from 80 percent in 
Portland to 85 percent in Atlanta. 

• Employment did not guarantee coverage: About 20 to 30 percent of individu- 
als in the control groups who were working did not have health coverage. 

Among those who were working and did have coverage, between 40 and 60 
percent obtained it from their employers. 

• Between 20 and 30 percent of adults in the control groups did not have health 
care coverage at the end of year 5. This indicates that many of those who left 
welfare for work were not able to replace Medicaid with private coverage 
once their transitional benefits expired. Control group respondents without 
coverage were just as likely as those with coverage to have been working at 
the end of the five-year follow-up, but they were much less likely to have 
been offered coverage through their jobs. 



'Mullahy and Wolfe, 2000. 
^Guyer and Mann, 1999. 
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• None of the programs had a statistically significant impact on coverage for 
adults or children. There is some evidence that because the programs in- 
creased employment, they may have led to a shift from public to private cov- 
erage. In all seven programs, program group respondents were more likely 
than those in the control group to have private health care coverage at the end 
of year 5 and less likely (in six of seven programs) to have publicly funded 
coverage, although these differences were not statistically significant. 

• Not all families eligible for Transitional Medicaid received it. Only about 60 to 
75 percent of control group members who left welfare for work received Tran- 
sitional Medicaid during the five-year follow-up period. Most of the programs 
increased the use of Transitional Medicaid, although only the impacts in River- 
side LFA and HCD were statistically significant. The effect in these two pro- 
grams is due to the fact that they increased the number of individuals who were 
eligible (those who left welfare for work) and slightly increased use among 
those who were eligible. 



II. Expected Effects 

At random assignment, all sample members and their children had health care coverage 
because they were receiving AFDC and thus automatically covered under Medicaid. In general, 
Medicaid receipt should fall over time for both the program and control groups since most of 
them left welfare during the five-year follow-up period. By the last quarter, for example, welfare 
receipt ranged from 20 to 40 percent across the sites. Health care coverage in general should also 
fall over time if some sample members cannot replace their lost Medicaid coverage. Coverage 
might also fall more rapidly for parents than for their children. Most children under age 18 are 
now eligible for Medicaid if their family's income is less than 100 percent of the poverty level. 
They may also be eligible, depending on the state in which they live, for coverage through the 
Children’s Health Insurance Program (CHIP). As a result, it is not uncommon for coverage status 
to vary within the same family. The mother might not have coverage or might be covered 
through her employer, while her children might be covered through CHIP or Medicaid. 

The primary way that the programs might affect health coverage is through their effects 
on welfare receipt and employment. When people leave welfare for work, they run the risk of 
losing coverage because they will either immediately or eventually lose their public coverage 
and cannot often find private coverage to replace it. Transitional Medicaid is the main source of 
public coverage and is available to families for up to one year after leaving welfare.^ However, 
recent research finds that its use has been slow to take hold: According to one study, over 
600,000 recipients lost Medicaid coverage when their welfare case was closed, even though the 
majority still met Medicaid eligibility standards.^* There are a number of possible reasons why 
individuals eligible for Medicaid do not receive it. One reason is that recently employed recipi- 
ents, believing they are no longer eligible for welfare, often fail to respond to eligibility verifica- 



^Kaplan, 1997. 

“'Families USA Foundation, 1999. 
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tion notices they receive from welfare caseworkers. Caseworkers typically close the cases of 
those who do not respond, which includes not only welfare but Medicaid and Food Stamps.^ 

Once Transitional Medicaid expires the family must find other insurance, mostly from 
private sources. This has become increasingly difficult for many low-income workers. There has 
been a decrease over the past decade in the number of workers with employer-sponsored cover- 
age, especially among those in low-wage jobs.^ With rising health care costs, many employers 
have been faced with the choice of dropping coverage for their employees or passing along the 
costs to them in the form of higher premiums. As a result, many former welfare recipients are 
either not offered insurance or cannot afford it. 

In terms of program impacts, increases in employment and reductions in welfare receipt 
might lead to reductions in coverage, particularly in those sites with the largest reductions in wel- 
fare receipt. Because better jobs are more likely to come with employer-provided insurance, pro- 
grams that may have encouraged recipients to take lower-quality jobs than they would have oth- 
erwise — particularly programs with an emphasis on quick employment — may have decreased 
coverage. In contrast, if program group members moved into better jobs than control group 
members over time, they may have increased coverage. Average wages, one measure of job 
quality, were higher for program group members in some sites (Atlanta LFA, Riverside HCD, 
and Riverside LFA all significantly increased hourly wages). Program impacts on earnings might 
also affect coverage, particularly coverage for children, since families with higher eamirigs may 
be less likely to qualify for public insurance programs. 

The programs might also affect coverage in ways that are not related to their impacts on 
employment or welfare receipt. For example, program group members have more contact than 
control group members with program staff, making it more likely that they will be made aware 
of and receive help in obtaining Transitional Medicaid or other kinds of public coverage. Pro- 
grams with a strictly enforced participation mandate might also affect coverage, probably reduc- 
ing it, if they lead many individuals to leave welfare before they have found jobs. 

III. Health Care Coverage at the End of Year 5 

Data on health coverage come from the Five-Year Client Survey. Since coverage is 
measured at only one point, the month before the five-year survey, it is not possible to measure 
the continuity of coverage. Respondents were considered covered if they reported receiving 
Medicaid, if they were working and had accepted their employer’s health insurance plan, or if 
they reported receiving coverage from another private source. If they were receiving welfare or 
SSI, they were assumed to be covered through Medicaid. 

A. Employment and Employer-Provided Coverage 

As discussed in Chapter 4, UI records data show that most of the programs began increas- 
ing employment within a year or two of random assignment, and the biggest effects were in River- 
side LFA and HCD, Grand Rapids LFA, and Portland. The employment impacts also lessened over 
time. By the end of year 5, only Riverside LFA and HCD and Portland continued to show signifi- 



^Quint and Widom, 2001. 
^Farber and Levy, 2000. 
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cant positive impacts. This pattern also holds with the survey data. The upper panel of Table 8.1 
shows impacts on employment at the time of the five-year survey. In Riverside LFA, Riverside 
HCD, and Portland, more individuals in the program group than the control group were working at 
the five-year point, although the difference in Portland is not statistically significant. 

The middle panel of Table 8.1 shows the percentage of program and control group mem- 
bers who were employed with health care coverage, either public or private. A comparison of 
this panel with the upper panel shows that a significant proportion of those working (ranging 
from 20 to 30 percent across sites) did not have health coverage. For example, 54 percent of con- 
trol group members in Atlanta were employed, but only 37 percent were employed and had cov- 
erage, meaning that about 70 percent of those working had coverage. Control group rates were 
highest in Riverside and Portland. In Riverside, the high welfare benefit levels meant that more 
welfee recipients could work and still remain eligible for some welfare and, therefore, Medi- 
caid. In Portland, the high coverage rate may reflect the existence of the state’s public health 
program — the Oregon Health Plan (OHP).® 

The lower panel of Table 8.1 shows the percentage of program and control group mem- 
bers who were employed with employer-provided coverage. Between 15 and 30 percent of con- 
trol group members were working and had employer-provided coverage. A comparison of these 
numbers with the upper and middle panels highlights several points. First, only about one-third 
to one-half of those who were employed had employer-provided coverage. In Atlanta, for exam- 
ple, 54 percent of the control group worked, but only 19 percent of the control group worked and 
had employer-provided coverage, meaning that only 35 percent of workers had coverage through 
their jobs. Second, 40 to 60 percent of workers with coverage had employer-provided coverage 
(compare the middle and lower panels), showing that other programs were an important source 
of coverage among low-wage workers. Other sources of coverage, for example, are Transitional 
Medicaid and coverage from a spouse (as shown in Chapter 9, from 10 to 20 percent of respon- 
dents were mamed at the five-year point). The percentage with employer-provided coverage is 
especially low in Riverside, where 40 percent of controls were working and had coverage, but 
only 15 percent had employer-provided coverage, meaning that 38 percent of insured workers 
had coverage through their jobs. 

One reason for the relatively low rates of employer-provided coverage is that some re- 
spondents were offered but declined to enroll in their employer’s plan. In Riverside and Atlanta, 
about 70 percent of those offered coverage accepted it compared with about 80 percent of those 
in Grand Rapids and Portland. The low employer-provided coverage rates in Riverside may re- 
flect the fact that relatively more workers were still eligible for welfare and Medicaid. Individu- 
als who have Medicaid are probably more likely to turn down an employer's offer of coverage. 

Only Riverside HCD produced a statistically significant increase in the number of re- 
spondents who were employed and had coverage (of any type) at the end of year 5. None of the 
programs had a statistically significant impact on being employed and having employer-provided 



In Riverside, about 16 percent of control group respondents (and one-third of those employed) combined 
work and welfare at the end of year 5, compared with less than 10 percent of control group members in the other 
three sites. 

The OHP provides publicly financed health care coverage to residents of the state of Oregon who do not qual- 
ify for Medicaid but whose income is below the poverty level. 
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National Evaluation of Welfare-to-Work Strategies 

Table 8.1 

Impacts on Employment and Health Care Coverage 



Site and Program 


Sample Program Control Difference Percentage 

Size Group (■%) Group (%) (Impact) Change (%) 






Emnloved at interview 




Atlanta Labor Force Attachment 


1,071 


57.2 


54.3 


2.9 


5.4 


Atlanta Human Capital Development 


1,146 


52.9 


54.3 


-1.4 


-2.5 


Grand Rapids Labor Force Attachment 


1,097 


65.1 


67.7 


-2.6 


-3.8 


Grand Rapids Human Capital Development 


1,109 


66.2 


67.7 


-1.5 


-2.2 


Riverside Labor Force Attachment 


1,219 


55.0 


48.9 


6.1 ♦♦ 


12.4 


Lacked high school diploma or basic skills 


657 


48.9 


43.0 


5.9 


13.6 


Riverside Human Capital Development 


778 


51.1 


43.0 


8.0 ♦♦ 


18.7 


Portland 


504 


61.7 


58.3 


3.4 


5.9 




Emnloved with health care coverage at interview 




Atlanta Labor Force Attachment 


1,071 


39.1 


36.9 


2.2 


6.0 


Atlanta Human Capital Development 


1,146 


38.0 


36.9 


1.1 


3.1 


Grand Rapids Labor Force Attachment 


1,097 


49.4 


51.8 


-2.4 


-4.6 


Grand Rapids Human Capital Development 


1,109 


51.9 


51.8 


0.1 


0.2 


Riverside Labor Force Attachment 


1,219 


40.9 


39.6 


1.4 


3.4 


Lacked high school diploma or basic skills 


657 


38.5 


33.8 


4.6 


13.7 


Riverside Human Capital Development 


778 


40.6 


33.8 


6.8 ♦ 


20.0 


Portland 


504 


47.8 


47.7 


0.1 


0.2 




Emnloved with emnlover-nrovided health 








care coverage at interview 




Atlanta Labor Force Attachment 


. 1,071 


22.2 


19.0 


3.2 


16.6 


Atlanta Human Capital Development 


1,146 


21.2 


19.0 


2.2 


11.5 


Grand Rapids Labor Force Attachment 


1,097 


30.5 


30.0 


0.5 


1.7 


Grand Rapids Human Capital Development 


1,109 


30.4 


30.0 


0.4 


1.5 


Riverside Labor Force Attachment 


1,219 


17.5 


15.0 


2.5 


16.6 


Lacked high school diploma or basic skills 


657 


12.0 


10.9 


1.1 


9.8 


Riverside Human Capital Development 


778 


14.8 


10.9 


3.9 


35.5 


Portland 


504 


31.6 


27.3 


4.4 


16.0 



SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A. 2. 
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coverage. However, the impacts on employer-provided coverage are similar in size to the im- 
pacts on employment in Atlanta, Grand Rapids, and Portland, although somewhat smaller in 
Riverside. This suggests that the program group members who were encouraged to work had 
jobs at the end of year 5 that were at least as likely to have insurance as the jobs held by control 
group members. 

B. Public Versus Private Coverage for Respondents 

Table 8.2 presents coverage more generally, not as it relates to employment status. Re- 
spondents were considered to be covered by public health insurance if they reported receiving 
Medicaid in the month prior to interview or if they were receiving welfare or SSI. They were 
considered to be covered by private health care if they had accepted their employer’s plan or had 
coverage from another private source. 

As the upper panel of Table 8.2 shows, coverage levels for the control group ranged from 
72 percent in Atlanta to 81 percent in Portland, meaning that 20 to 30 percent of respondents 
were uninsured at the end of year 5. The percentage uninsured is higher than it is for the nation 
as a whole but lower than it is for low-income individuals. A recent study reported that 16 per- 
cent of all adults with children were uninsured; among those with incomes below poverty, 42 
percent were uninsured.^ A later section in this chapter will look more closely at the respondents 
who no longer had coverage at the time of the survey. 

The middle and lower panels of Table 8.2 show that the majority of the coverage is from 
public sources. In Portland, for example, 80 percent of control group members had any coverage, 
47 percent had public coverage, and 38 percent had private coverage. This extent of public cov- 
erage makes sense, considering that 20 to 40 percent of sample members were still on welfare at 
the five-year point and that those who left welfare shortly before that were probably still receiv- 
ing Transitional Medicaid (data on the take-up of Transitional Medicaid are shown in a later sec- 
tion). In Riverside, for example, nearly 40 percent of control group members were still receiving 
welfare at the end of year 5, which explains why the extent of public coverage is relatively high 
in this site. In Portland, the extent of public coverage is also related to Oregon’s OHP, since 
about 20 percent of control group members were receiving welfare at the five-year point. 

None of the programs produced a statistically significant impact on coverage (upper 
panel of Table 8.2). Levels of coverage were about the same for program group members as for 
control group members. This result is encouraging since the expectation was that many welfare 
recipients would lose insurance as they went from welfare to work. 

The impacts on types of coverage (middle and lower panels of Table 8.2) show that the 
programs may have led to a shift from public to private sources. Although few of the impacts are 
statistically significant, the general pattern is that program group members were less likely to 
have public coverage and more likely to have private coverage than control group members. For 
the Atlanta LFA program, this shift in coverage is statistically significant; the program reduced 
public health care coverage by 5 percentage points and increased private coverage by 5 percent- 
age points. This is consistent with the program’s moving more welfare recipients to work. 



^See Holahan and Brennan, 2000. 
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National Evaluation of Welfare-to-Work Strategies 
Table 8.2 

Impacts on Health Care Coverage for Respondents at the End of Year 5 



Site and Program 


Sample 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 






Has health care coverage 




Atlanta Labor Force Attachment 


1,071 


71.0 


72.4 


-1.4 


-1.9 


Atlanta Human Capital Development 


1,146 


74.0 


72.4 


1.6 


2.2 


Grand Rapids Labor Force Attachment 


1,097 


. 75.1 


77.7 


-2.6 


-3.3 


Grand Rapids Human Capital Development 


1,109 


77.8 


77.7 


0.1 


0.2 


Riverside Labor Force Attachment 


1,219 


78.3 


80.3 


-2.0 


-2.5 


Lacked high school diploma or basic skills 


657 


82.7 


80.0 


2.7 


3.4 


Riverside Human Capital Development 


778 


80.3 


80.0 


0.3 


0.4 


Portland 


504 


74.7 


80.6 


-6.0 


-7.4 






Has public health care coverage 




Atlanta Labor Force Attachment 


1,071 


46.5 


51.7 


-5.2 * 


-10.1 


Atlanta Human Capital Development 


1,146 


50.3 


51.7 


-1.4 


-2.7 


Grand Rapids Labor Force Attachment 


1,097 


41.2 


42.5 


-1.2 


-2.9 


Grand Rapids Human Capital Development 


1,109 


43.2 


42.5 


0.7 


1.7 


Riverside Labor Force Attachment 


1,219 


55.8 


59.4 


-3.6 


-6.1 


Lacked high school diploma or basic skills 


657 


65.3 


66.4 


-1.1 


-1.7 


Riverside Human Capital Development 


778 


63.4 


66.4 


-3.1 


-4.6 


Portland 


504 


43.2 


47.0 


-3.7 


-7.9 






Has private health care coverage 




Atlanta Labor Force Attachment 


1,071 


28.6 


24.1 


4.5 * 


18.5 


Atlanta Human Capital Development 


1,146 


27.4 


24.1 


3.3 


13.5 


Grand Rapids Labor Force Attachment 


1,097 


41.8 


40.7 


1.1 


2.7 


Grand Rapids Human Capital Development 


1,109 


41.3 


40.7 


0.5 


1.3 


Riverside Labor Force Attachment 


1,219 


26.9 


25.7 


1.2 


4.7 


Lacked high school diploma or basic skills 


657 


20.9 


18.0 


2.9 


16.4 


Riverside Human Capital Development 


778 


23.9 


18.0 


5.9 * 


33.1 


Portland 


504 


39.0 


37.8 


1.2 


3.2 



SOURCE: MDRC calculations from the Five-Year Client Survey. 



NOTES: See Appendix A.2. 



C. Coverage for Children 



Table 8.3 presents coverage for respondents’ children. Respondents reported the coverage 
status of each of their children at the end of year 5. This analysis is limited to respondents’ de- 
pendent children aged 18 or younger at the five-year interview date.'° Children were considered 
to be covered if respondents reported that children had coverage from Medicaid or from a private 
insurer or if respondents reported receiving welfare or SSI benefits. Although data were collected 
for each child, the following analysis examines whether all children in the family were covered." 

A comparison of Table 8.3 with Table 8.2 shows that children were somewhat more 
likely than adults to have health care coverage at the end of year 5. In Grand Rapids, for exam- 
ple, 82 percent of control group children were covered compared with 78 percent of respondents. 
This is perhaps not surprising given the range of public programs that specifically cover low- 
income children. Nonetheless, the rates of noncoverage (15 to 20 percent) suggest that these pro- 
grams are not serving all eligible children. 

The greater number of public programs for children also is reflected in the fact that chil- 
dren were more likely than adults to be covered through public sources and less likely through 
private sources. Only 33 percent of control group children in Grand Rapids, for example, had 
private insurance compared with 41 percent of adults. Employers may not have offered to cover 
children or the adults may have turned down this coverage if it was too expensive. 

Finally, none of the programs had statistically significant effects on coverage for chil- 
dren. They did lead to a slight shift from public to private sources, which mirrors the effects 
found for adults. It is encouraging that the programs, by leading more families from welfare to 
work, did not negatively affect children’s health care coverage. 

IV. Transitional Medicaid Use During the Five-Year Follow-TJp 

For many welfare recipients, the potential loss of health coverage can create a disincen- 
tive to leave welfare and go to work. Transitional Medicaid reduces this disincentive by allowing 
working individuals to continue coverage, for themselves and their children, for up to one year 
after leaving welfare.’^ Receipt of this benefit is not automatic, however, and eligible recipients 
must request it and receive caseworker approval.'^ Data on the receipt of Transitional Medicaid 
are available from the survey: Respondents were asked whether they had ever received this bene- 
fit during the five-year follow-up period. 

Table 8.4 shows the percentage of respondents who were eligible for and used Transi- 
tional Medicaid. The upper panel shows the percentage of program and control group members 



'®About 8 percent of respondents were excluded from this analysis either because none of their children was 18 
or under at interview or because they reported having no children. 

"in very few families — between 3 and 7 percent of control group families — were only some (but not all) of 
the children in the family covered. 

In 1998 the State of California increased Transitional Medicaid coverage for up to two years. This change 

would have affected only the sample members in Riverside who entered the program near the end of the random 
assignment period. 

'^Recipients who find jobs and stop communicating with caseworkers might lose transitional coverage, since 
caseworkers might close their cases. 
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National Evaluation of Welfare-to-Work Strategies 
Table 8.3 

Health Care Coverage for Respondents' Children at the End of Year 5 



Site and Program 


Sample Program Control Difference Percentage 

Size Group (%) Group (%) (Impact) Change (%) 




All denendent children have health care coverage 


Atlanta Labor Force Attachment 


974 


85.1 


84.5 


0.6 


0.7 


Atlanta Human Capital Development 


1,057 


83.4 


84.5 


-1.1 


-1.3 


Grand Rapids Labor Force Attachment 


1,005 


78.8 


81.8 


-3.0 


-3.6 


Grand Rapids Human Capital Development 


1,026 


79.3 


81.8 


-2.5 


-3.1 


Riverside Labor Force Attachment 


1,120 


81.9 


83.2 


-1.3 


-1.6 


Lacked high school diploma or basic skills 


614 


83.8 


82.1 


1.8 


2.2 


Riverside Human Capital Development 


743 


85.2 


82.1 


3.2 


3.8 


Portland 


451 


75.5 


80.2 


-4.7 


-5.8 






All dependent children have 








public health care coverage 




Atlanta Labor Force Attachment 


974 


63.7 


69.1 


-5.3 * 


-7.7 


Atlanta Human Capital Development 


1,057 


66.6 


69.1 


-2.5 


-3.6 


Grand Rapids Labor Force Attachment 


1,005 


49.7 


51.3 


-1.6 


-3.2 


Grand Rapids Human Capital Development 


1,026 


52.1 


51.3 


0.8 


1.5 


Riverside Labor Force Attachment 


1,120 


57.6 


63.9 


-6.2 ** 


-9.8 


Lacked high school diploma or basic skills 


614 


66.0 


70.9 


-4.9 


-6.9 


Riverside Human Capital Development 


743 


67.8 


70.9 


-3.1 


-4.4 


Portland 


451 


44.1 


52.1 


-8.0 


-15.3 






All dependent children have 








private health care coverage 




Atlanta Labor Force Attachment 


974 


19.8 


16.0 


3.9 * 


24.2 


Atlanta Human Capital Development 


1,057 


16.0 


16.0 


0.0 


0.1 


Grand Rapids Labor Force Attachment 


1,005 


31.5 


32.8 


-1.3 


-4.0 


Grand Rapids Human Capital Development 


1,026 


29.2 


32.8 


-3.6 


-10.8 


Riverside Labor Force Attachment 


1,120 


22.9 


19.9 


3.1 


15.5 


Lacked high school diploma or basic skills 


614 


16.4 


12.4 


4.0 


32.3 


Riverside Human Capital Development 


743 


19.1 


12.4 


6.7 ** 


54.2 


Portland 


451 


30.2 


28.6 


1.7 


5.8 



SOURCE: MDRC calculations from the Five-Year Client Survey. 



NOTES: See Appendix A. 2. 
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National Evaluation of Welfare-to-Work Strategies 
Table 8.4 

Impacts on Transitional Medicaid Benefits 



Site and Program 


Sample Program Confrol 

Size Group (%) Group f%) 


Difference Percentage 

(Impact) Change (%) 




Ever emnloved and off welfare during follow- 


012 


Atlanta Labor Force Attachment 


1,071 


59.7 


56.2 


3.5 


6.2 


Atlanta Human Capital Development 


1,146 


59.6 


56.2 


3.3 


5.9 


Grand Rapids Labor Force Attachment 


1,097 


71.3 


67.7 


3.6 


5.4 


Grand Rapids Human Capital Development 


1,109 


67.9 


67.7 


0.3 


0.4 


Riverside Labor Force Attachment 


1,219 


46.9 


39.6 


7.2 *** 


18.3 


Lacked high school diploma or basic skills 


657 


37.7 


28.3 


9 4 ♦♦♦ 


33.3 


Riverside Human Capital Development 


778 


36.6 


28.3 


8.2 ** 


29.1 


Portland 


504 


67.8 


66.8 


1.0 


1.6 




Ever covered bv Transitional Medicaid 








during follow-uo 




Atlanta Labor Force Attachment 


1,071 


42.0 


38.1 


3.8 


10.1 


Atlanta Human Capital Development 


1,146 


42.7 


38.1 


4.6 


12.1 


Grand Rapids Labor Force Attachment 


1,097 


53.5 


50.8 


2.7 


5.3 


Grand Rapids Human Capital Development 


1,109 


50.2 


50.8 


-0.6 


-1.2 


Riverside Labor Force Attachment 


1,219 


32.3 


25.8 


6.5 ** 


25.3 


Lacked high school diploma or basic skills 


657 


25.6 


17.9 


'7 '7 ♦♦ 


42.8 


Riverside Human Capital Development 


778 


26.2 


17.9 


8.2 ** 


45.9 


Portland 


504 


55.5 


50.8 


4.6 


9.1 




For those ever off welfare and emnloved. 








ever covered bv Transitional Medicaid 




Atlanta Labor Force Attachment 




.70.3 


67.8 


2.5 


3.6 


Atlanta Human Capital Development 




71.7 


67.8 


4.0 


5.8 


Grand Rapids Labor Force Attachment 




75.0 


75.1 


-0.1 


-0.1 


Grand Rapids Human Capital Development 




73.9 


75.1 


-1.2 


-1.6 


Riverside Labor Force Attachment 




68.8 


65.0 


3.9 


6.0 


Lacked high school diploma or basic skills 




67.9 


63.3 


4.6 


7.2 


Riverside Human Capital Development 




71.6 


63.3 


8.2 


13.0 


Portland 




81.8 


76.2 


5.7 


7.4 



SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A.2. 
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who worked and were off welfare at some point during the follow-up period and indicates the 
number of respondents who would have been eligible to receive Transitional Medicaid. Between 
56 and 68 percent of control group members worked and left welfare during the follow-up period 
in Atlanta, Grand Rapids, and Portland. A much smaller percentage of controls (40 percent) were 
eligible for Transitional Medicaid in Riverside, probably because fewer in this site worked and 
when they did many were still eligible for welfare. Both programs in Riverside produced a sig- 
nificant increase in the percentage working and off welfare. 

The middle panel of Table 8.4 shows that not all who were eligible received Transitional 
Medicaid. This can be seen more easily in the lower panel, which shows receipt of Transitional 
Medicaid by those who were eligible (this is a nonexperimental comparison). Data for the con- 
trol groups show that about 65 to 76 percent of eligible respondents received this benefit. In At- 
lanta, for example, 56 percent of control group members were eligible for Transitional Medicaid 
and 38 percent received it, meaning that only 68 percent of eligible respondents received it. Data 
from the two-year survey (not shown) indicate that the use of this benefit increased over time, 
probably as more families became eligible. At the two-year point, between 10 and 25 percent of 
control group members reported having used Transitional Medicaid compared with a much 
higher percentage at the end of year 5. 

The programs generally increased Transitional Medicaid use — although only Riverside 
LFA and Riverside HCD produced statistically significant increases — because they increased 
the percentage who were eligible to receive it, but also because they increased its use among 
those eligible. The similarity of the impacts on the number of respondents who were employed 
and off welfare and on the number who used Transitional Medicaid suggests that most of those 
individuals who were induced by the program to work and leave welfare did, in fact, receive 
Transitional Medicaid at some point. This is also reflected in the nonexperimental numbers in the 
lower panel of Table 8.4. Of program group members who were eligible in Portland, for exam- 
ple, 82 percent received the benefit compared with only 76 percent of control group members. 
This finding is consistent with one of the hypotheses raised earlier that more involvement by 
caseworkers would lead to greater use of available benefits. 

V. Loss of Coverage bv the End of Year 5 

Section III of this chapter showed that 20 to 30 percent of respondents no longer had 
health coverage at the end of year 5. Were these families less likely to work, did they work in 
lower-quality jobs, or did they earn too much to qualify for public programs? Although an exten- 
sive analysis of the uninsured is beyond the scope of the chapter, this section presents a compari- 
son of adults who had coverage and those who did not have coverage at the end of the five-year 
follow-up. 

Figure 8.1 presents coverage status for respondents at the end of year 2 and at the end 
of year 5.’^* The figure shows that between 70 and 80 percent of respondents had coverage at 
the end of year 5. Also, among those who were uninsured at that point, most had lost coverage 
after year 2. 



*'*For more detailed information about health care coverage at the end of year 2, see Freedman et al., 2000a, 
Chapter 8. 
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Figure 8.1 

Health Care Coverage for Respondents Over Five Years 
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SOURCES: MDRC calculations from the Two-Year Client Survey and Five-Year Client Survey. 









Table 8.5 compares uninsured and insured control group members. The first two rows 
show that the uninsured were equally if not more likely to have been working at the end of the 
five-year follow-up than the insured, but they were much less likely to have been offered cover- 
age by their employers. In Grand Rapids, for example, 69 percent of the uninsured were working 
at the end of the follow-up, but only 20 percent were working and were offered employer- 
provided coverage. In contrast, 66 percent of insured control group members were working at the 
end of the follow-up and 43 percent of them were offered coverage through their jobs. 

The uninsured on average earned less than the insured in the last quarter of follow-up. 
This difference is consistent with the lower rate of employer-provided coverage and suggests that 
the uninsured may have been working in lower-quality jobs, including more part-time work. The 
lower earnings also suggest that the lack of coverage is not due to the fact that these workers 
were earning too much to qualify for public programs serving low-income families. In Portland, 
for example, average earnings in the last quarter of year 5 were $1,165 for the uninsured, which 
is less than $5,000 on an annual basis. The OHP provides coverage to families who are not eligi- 
ble for Medicaid but have incomes below the poverty line. Although coverage rates were gener- 
ally higher in Portland than in the other sites (see Table 8.2), many eligible people probably did 
not receive benefits. The low earnings also suggest that the children of uninsured respondents 
were probably eligible for Medicaid or other public health programs. Nonetheless, only between 
one-third and one-half of them were covered. In Riverside, for example, only 33 percent of unin- 
sured adults had coverage for their children. 

Finally, the uninsured were at least as likely as the insured to have received Transitional 
Medicaid during the follow-up period. This suggests that they were not more likely to immedi- 
ately lose coverage when they made the transition from welfare to work, but that they were un- 
able to replace Medicaid when their transitional benefits expired. 

VI. Conclusions 

A potential effect of the NEW WS programs was the loss of health coverage for partici- 
pants as they moved from welfare to work. Many low-wage workers either are not offered or do 
not accept employer-provided insurance, making it difficult to replace their lost Medicaid with 
private coverage. It is encouraging that the programs did not lead to a loss in coverage for the 
adults or their children. 

As expected, levels of health care coverage decreased for both program and control 
group members over time, though a large majority of respondents and their children were still 
insured at the end of the five-year follow-up. Since the evaluation started, states have contin- 
ued to address the issue of coverage for low-income families by creating or expanding public 
programs. These additional benefits were not a program component and were available to 
members of both research groups. The findings presented in this chapter indicate that the sites 
were generally successful in continuing to provide coverage for families who left welfare. 
Nonetheless, 20 to 30 percent of adults were not covered, suggesting that states should con- 
tinue in their efforts to reach the uninsured. Because access to health care has been expanding, 
the findings presented here on the effects of the programs might have been different had the 
evaluation taken place in more recent years. 





..'n 



- 171 - 



216 



National Evaluation of Welfare-to-Work Strategies 



Table 8.5 

Characteristics of Control Group Members With and Without 
Health Care Coverage at the End of Year 5 



Site 


Respondent 
Covered at End 
of Year 5 


Respondent Not 
Covered at End 
of Year 5 


Atlanta 






Employed at end of year 5 (%) 


54.4 


67.8 


Employed and offered coverage from employer (%) 


36.5 


19.9 


Earnings in last quarter ($) 


1,729 


1,597 


Ever received Transitional Medicaid (%) 


39.4 


50.7 


Covered at end of year 2 (%) 


92.1 


83.8 


All children covered at end of year 5 (%) 


95.5 


54.3 


Sample size 


406 


146 


Grand Rapids 






Employed at end of year 5 (%) 


65.7 


68.9 


Employed and offered coverage from employer (%) 


42.7 


19.7 


Earnings in last quarter ($) 


2,077 


1,365 


Ever received Transitional Medicaid (%) 


48.2 


54.1 


Covered at end of year 2 (%) 


89.2 


77.7 


All children covered at end of year 5 (%) 


90.9 


48.2 


Sample size 


440 


122 


Riverside 






Employed at end of year 5 (%) 


49.4 


47.5 


Employed and offered coverage from employer (%) 


25.4 


9.2 


Earnings in last quarter ($) 


1,181 


1,059 


Ever received Transitional Medicaid (%) 


26.4 


22.7 


Covered at end of year 2 (%) 


92.2 


76.5 


All children covered at end of year 5 (%) 


94.2 


32.5 


Sample size 


579 


141 


Portland 






Employed at end of year 5 (%) 


60.8 


59.5 


Enployed and offered coverage from employer (%) 


43.1 


9.5 


Earnings in last quarter ($) 


1,745 


1,165 


Ever received Transitional Medicaid (%) 


50.3 


50.0 


Covered at end of year 2 (%) 


92.1 


85.0 


All children covered at end of year 5 (%) 


86.5 


43.2 


Sample size 


181 


42 



SOURCE: MDRC calculations from the Five-Year Client Survey. 



NOTES: See Appendix A. 2. 
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Chapter 9 

Impacts on Household and Personal Circumstances 

This chapter examines the effects of employment- and education-focused programs on a 
variety of outcomes meant to capture the household and personal circumstances of families in the 
NEWWS Evaluation, such as sample members’ household composition, relationships with other 
household members and with people outside the household (such as colleagues at work), and 
housing status. 

Policymakers and researchers have increasingly come to recognize that welfare recipi- 
ents’ household and personal circumstances may influence the long-term impacts of welfare-to- 
work programs as well as mediate the impacts of these programs on children. For example, as 
welfare recipients go to work or increase their hours of employment, a spouse, partner, or ex- 
tended family member may provide important support, particularly by caring for young children. 
The financial contribution of other members of the family or household may also play an essen- 
tial role in moving a family out of poverty and into long-term self-sufficiency — particularly 
since, as demonstrated in earlier chapters, these programs do not necessarily increase respon- 
dents’ total income. The quality of personal and other relationships and housing status can also 
influence employment and other economic outcomes. For instance, welfare recipients who must 
contend with verbal or physical threats or who are worried about leaving their children alone at 
home in an unsafe neighborhood may have difficulty getting a job and keeping it. 

At the two-year follow-up, few of the 1 1 welfare-to-work programs evaluated were found to 
have effects on marriage or additional births (that is, the number of children bom since random as- 
signment).' This chapter examines whether these programs produced impacts on these and related 
outcomes five years after study entry. In addition, for six programs in three sites, it examines pro- 
gram impacts on measures not examined at the two-year point — specifically, measures of physical 
and nonphysical employment-related and domestic abuse by intimate partners or others. 

Earlier chapters in this report presented impacts on employment, earnings, welfare re- 
ceipt, and income for the full sample in the evaluation. This chapter examines outcomes and im- 
pacts for the Five-Year Client Survey sample, which included sample members in Atlanta, Grand 
Rapids, Riverside, and Portland (for details, see Chapter 2). In general, as discussed in Appendix 
I, the pattern of impacts for the client survey sample and the administrative records sample were 
similar for the LFA and HCD programs in Atlanta and Grand Rapids. The Riverside LFA pro- 
gram produced larger reductions in welfare and, consequently, larger losses in income for the cli- 
ent survey sample than for the full sample, and the Riverside HCD program produced larger in- 
creases in earnings, and, consequently, smaller losses in income for the client survey sample than 
for the full sample. The effects of the Portland program were also much less positive for the cli- 
ent survey sample than for the full sample. Thus, generalizations concerning how the Portland 
program effects on economic outcomes, especially those documented in earlier chapters, may 
have affected the outcomes described in this chapter should be made with considerable caution. 



'Freedman et al., 2000a, Appendix C. 
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Furthermore, it is important to remember that many of the family outcomes examined in this 
chapter were measured at the time of the five-year survey or during the last year of follow-up, by 
which time the control group embargo had been lifted in Atlanta and Grand Rapids. 

The outcomes presented in this chapter depict important aspects of sample members’ 
lives — as a representative subsample of welfare recipients — and, thus, are noteworthy whether 
or not program impacts occurred. For this reason, hypotheses are followed by a descriptive analy- 
sis of selected outcomes that are of particular interest in terms of welfare and employment policy: 
marriage, living with unrelated adults and reports of domestic abuse. The chapter discusses pro- 
gram impacts and, briefly, links between effects on these aspects of sample members’ lives and 
program practices and program effects on employment. 

I. Key Findings 

These welfare-to-work programs generally had few effects on household and personal 
circumstances. The lack of dramatic changes in outcomes such as marriage and fertility are not 
surprising since these programs were structured primarily to alter employment behavior rather 
than aimed at decreasing additional births or affecting marriage. Even so, the lives of program 
participants were affected in ways that are reassuringly positive: Some program group members 
moved to get better housing, one program increased home ownership, and nearly all LFA and 
HCD programs reduced reports of physical abuse. Furthermore, there were more effects on these 
latter outcomes than would be expected by chance. All program effects on household and per- 
sonal circumstances are summarized in Table 9.1. 

• Employment- and education- focused programs produced no signiflcant 
effects on marriage. However, there is some evidence that program group 
members engaged in less-formal relationships. The Riverside LFA program 
increasing cohabitation by 4.6 percentage points (or by 43 percent). All other 
programs showed a pattern of increased rates of cohabitation, though the ef- 
fects of these other programs were not statistically significant. 

• There were few program impacts and no differences in effects by pro- 
gram approach on measures of fertility and household composition. 

• Program impacts on moving and housing were concentrated in Grand 
Rapids. Program group members in both Grand Rapids programs were more 
likely to move — and more than once — since random assignment, primarily 
for better housing. The Grand Rapids HCD program also increased home 
ownership at the five-year follow-up. There were no significant differences be- 
tween program approaches on these outcomes. 

• Although there were few program impacts on many aspects of the quality 
of relationships, such as ever experiencing employment-related discour- 
agement, harassment or deterrence, or nonphysical abuse, there were 
fewer reports of experiencing any physical abuse during the last year of 
follow-up for program group members than for control group members. 

A similar proportion of program and control group members reported experi- 
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Table 9,1 



Summary of Impacts on Household and Personal Circumstances 





Atlanta 


Grand Rapids 




Riverside 








LFA 


LFA 




LFA HCD 


LFA HCD 


Full 


In-need^ HCD Portland 


Marital status^ 










Married 










Cohabiting 






I 


I 


Separated or divorced‘s 




D 






Never married 










Fertility** 










Presence of a new baby 








I 


Household composition*^ 










Lives alone 


D D 






I 


Lives with other adult and/or 










children‘s 










Moving and housing status 










Ever moved** 




I I 






Moved more than once** 




I I 


I 


I 


Owns a home^ 




I 






, Other housing*’’ ^ 


I 








Domestic abuse in prior year®’*’ 


D 


D 


D 


D 



NOTES: "I” indicates that the program produced a statistically significant increase in the outcome. "D" 
indicates a statistically significant decrease in the outcome. Blank spaces indicate that there were no impacts. 

^Sample members lacked a high school diploma or basic skills. 

^Measured in the month prior to the five-year survey interview date. 

^Measure includes the few incidences of widowhood. 

‘^Responses reflect the time period between random assignment and the five-year follow-up interview date. 

‘^Outcomes examined in this category include lives with children only, lives with partner/spouse and 
children, lives with adult extended family and any children, and lives with unrelated adults. 

^Includes persons who may have lived in public/subsidized housing, rented home or room, group home, 
shelter, jail, or other temporary housing facilities. 

^Measured in the year prior to the five-year follow-up interview for the Child Outcomes Study sample in 
Atlanta, Grand Rapids, and Riverside. 

^Other outcomes that were evaluated include any lifetime experience of employment-related or domestic 
abuse. Grand Rapids LFA decreased ever experiencing job deterrence and increased ever experiencing any 
abuse or nonphysical abuse. 
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encing harassment, abuse (physical or otherwise), or other types of employ- 
ment-related discouragement or deterrence at any time in their lives. However, 
there were fewer reports of physical abuse, by about 4 to 6 percentage points, 
during the last year of follow-up for program group members than for control 
group members in all sites, with these effects being statistically significant in 
the Altanta LFA, Grand Rapids HCD, and Riverside LFA programs. There is 
some evidence that program effects on increasing employment, especially ef- 
fects on employment that occurred early in the follow-up period (for example, 
by increasing self-esteem, ameliorating family stress, or simply reducing the 
amount of time individuals spent with partners), and enhanced caseworker at- 
tention to support services contributed to the fewer reports of experiencing 
physical abuse among program group members. These effects were measured 
only for mothers of children who were preschool-age at study entry in the six 
LFA and HCD programs (the Child Outcomes Study sample). 

II. Measurement Issues 

The outcomes examined in this chapter generally fall into the following categories: mari- 
tal status, household composition, moving and housing status, and employment-related and do- 
rnestic abuse by intimate partners and others. The relevant time periods for which these outcomes 
are measured also vary, with some capturing circumstances at the time of the survey interview, 
the month prior to the interview, during the last year of the follow-up, during the entire five-year 
follow-up period, or at any point during an individual’s life. 

A. Marital Status and Household Composition 

Marital status was measured at one point in time: the month prior to the survey interview. 
Dynamic or cumulative patterns of marital status are therefore not captured. Five mutually exclu- 
sive categories of marital status were created: married and living with a spouse, cohabiting, sepa- 
rated, divorced, widowed, and never married. 

Measures of household composition were constructed from a grid that questioned the re- 
spondent about all household members who stayed in her home for at least two nights a week 
during the month prior to the survey interview. For all measures of household composition, it is 
assumed that household members in the respondent’s generation or older are adults^ and that 
household members in a younger generation are children.^ Adult extended family members in- 
clude parents, siblings, grandparents, aunts and uncles, cousins, and other adult relatives. For 
comparability with census definitions of “living with unrelated adults,” the measure of living 
with unrelated adults was constructed based on information about household members and mari- 
tal status.'* If respondents listed unrelated adults as household members or reported not being 



^This category includes spouse, partner, father, mother, brother, sister, grandfather, grandmother, uncle, aunt, 
cousin, other adult male or female relative, and other unrelated male or female adult. 

^This category includes son, daughter, nephew, niece, grandson, granddaughter, and other unrelated male or fe- 
male child. 

‘*U.S. Bureau of the Census, 2000. 



married and living as a couple with a boyfriend or partner, they were considered to be living with 
unrelated adults. “Living with unrelated adults” includes cohabiting partners who may have fa- 
thered at least one child in the household.^ All marital status categories were coded independ- 
ently from household composition outcomes. For example, some respondents could have re- 
ported living with a “partner” or a “spouse” but may not have reported being “married” or “co- 
habiting.”^ Specific information about whether or not the father of at least one child lived in the 
household was not collected for all client survey respondents. Thus, respondents who reported 
being married or cohabiting may include marriage or cohabitation with the father of at least one 
child in the household. Presence of a new baby during the five-year follow-up period was meas- 
ured by comparing the birth dates of each biological, legally adopted, or step child with the date 
of random assignment (plus nine months). To evaluate changes in outcomes over time and com- 
pare these patterns with national figures (described in the final section of this chapter), compara- 
ble outcomes were created for sample members who had information collected at both the two- 
year follow-up and the five-year follow-up. 

B. Moving and Housing Status 

Measures of moving reflect any moves that occurred during the five-year follow-up pe- 
riod. The proportion of respondents who moved is calculated as the number who reported mov- 
ing at least once since the time of random assignment over all respondents. Measures of housing 
status reflect only status during the month prior to the survey interview. Respondents were con- 
sidered to be renting a home or room if they reported renting their own home or room or living 
with family or friends and contributing to part of the rent. Respondents were considered to be 
living in other housing arrangements if they reported living in a group shelter, in jail, alone and 
rent free, in another unspecified housing arrangement, or being homeless. Living in public or 
subsidized housing includes living in housing owned or operated by a local housing authority or 
other government agency as well as Section 8 housing. 

C. Employment-Related and Domestic Abuse 

Measures of employment-related and domestic abuse were collected at the five-year fol- 
low-up for sample members who were part of the Child Outcomes Study^ via a self-administered 
questionnaire, a method intended to preserve the privacy of the interviewee and increase valid 
response rates to sensitive questions. Average rates of nonresponse on these outcomes ranged 
from about 4 to 10 percent for questions about employment-related abuse and from about 7 to 15 



^Including cohabitors who may have fathered at least one child in the household is consistent with recent work 
by Primus et al., 1999, that documents the proportion of single mother families who live with unrelated adults using 
data from the Current Population Survey. Information about whether or not a father lives in the household is avail- 
able for a subset of client survey respondents (those who were also in the Child Outcomes Study sample). However, 
even this information is relevant only for the focal child in the family. 

^The percentage of respondents who reported these types of discrepancies is small: 2.6 percent of respondents 
reported being married or cohabiting and did not list a spouse or partner as a household member; 0.7 percent of re- 
spondents reported not being married or cohabiting but listed a spouse or partner as a household member. 

’The incidences of each of these categories’ alone were negligible — less than 5 percent of the client survey 
sample reported living in any one of these arrangements. 

®For more detailed information on the Child Outcomes Study sample, see Chapter 12. 



percent for more specific questions about domestic abuse.^ Multiple measures depicting the qual- 
ity of employment-related and other relationships were created, including job-related abuse that 
occurred at any time in a respondent’s life, abuse (physical or nonphysical) by intimate partners 
or others at any time in a respondent’s life, and abuse that occurred during the year prior to the 
five-year follow-up interview. 

Employment-related abuse (including job discouragement, job harassment, and job deter- 
rence) is measured over a respondent’s lifetime (though individuals are likely to have experi- 
enced these employment-related aspects of abuse only as an adult when they were actually em- 
ployed). Respondents were considered to have experienced job discouragement at any time in 
their life if they reported at least one of the following: someone ever tried to discourage them 
from finding a job or going to work; someone ever made them feel guilty about going to work; 
someone ever refused to help them or went back on promises to help with child care, transporta- 
tion, or housework; someone ever made it difficult for them to attend or complete programs or 
classes that would help them get a good job. The measure of job harassment includes being har- 
assed at the workplace over the telephone and/or in person. Respondents were considered to have 
experienced job deterrence at any time in their life if they reported that someone had ever caused 
them to quit or lose a job and/or someone ever prevented them from finding a job. These vari- 
ables were coded as “0” for respondents reporting that they did not experience job discourage- 
ment, harassment over the phone or in person, or job deterrence. 

Because individuals in program and control groups are similar at the time of random as- 
signment in their observed and unobserved characteristics, including experience with abuse prior 
to random assignment, it is likely that program impacts on these “lifetime” measures will capture 
effects that occurred during the five-year follow-up period. Furthermore, program group mem- 
bers may have been more likely to experience these types of employment-related abuse during 
this period since these welfare-to-work programs targeted and increased employment. As a result, 
fewer reports of employment-related abuse for program group members than for control group 
members will be especially difficult to detect. Nonetheless these measures are important in pro- 
viding a general picture of rates of employment-related abuse in welfare populations. 

Measures of experiencing any physical or nonphysical domestic abuse were created for 
two points in time: at any time in the respondent’s life and during the year prior to the five-year 
follow-up interview. Information about the timing of the most recent incident of abuse (for ex- 
ample, this week, a week ago, a month ago, six months ago, a year ago, or more than a year ago) 
was collected only for those respondents who reported any domestic abuse. Respondents were 
generally asked if they had experienced any of the following types of domestic abuse: being 
yelled at or put down on purpose, controlled, insulted, or sworn at; threatened with physical 
harm; hit, slapped, kicked, or otherwise physically harmed; or if “none of these things have ever 
happened to me.” Measures of more recent domestic abuse were also created for those individu- 



®Gallup-Black, 1999, discusses in detail the quality of data on domestic abuse and domestic barriers to work 
when collected via self-administered questionnaire (SAQ) and Computer-Assisted Self-Interview (CASI). CASI us- 
ers sit alone with a computer and headphones and type their answers directly into the computer. Gallup-Black com- 
pared SAQ-based NEWWS data with CASI-based Minnesota Family Investment Program (MFIP) data. She found 
that NEWWS respondents were less likely to answer questions about sensitive items than MFIP respondents. 
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als who reported these same types of abuse in the year prior to the survey interview (for example, 
the most recent incident happened either the week of the survey interview or the week, month, 
six months, or year prior to the survey interview). For these measures of recent abuse, respon- 
dents who did not report any domestic abuse or who reported that the most recent incident oc- 
curred more than a year ago were coded as “0.” The measures of more recent physical domestic 
abuse include being hit, slapped, kicked, or otherwise physically harmed; the measures of more 
recent nonphysical domestic abuse include all other types of abuse. Measures of physical abuse 
and nonphysical abuse are not mutually exclusive. 

III. Effects of Welfare-to-Work Approaches on Marital Status 

The policies followed in the welfare-to-work programs examined in this report are not 
expected to directly affect, in this case increase or preserve, the likelihood of marriage. Given 
the effects on economic outcomes such as employment, earnings, and income, how may these 
welfare-to-wOrk programs affect the marital status of survey respondents? Increased employment 
may increase the likelihood of marriage by expanding a single mother’s social network or in- 
creasing her self-esteem, and, perhaps, her attractiveness to a potential partner. Or increased em- 
ployment may decrease the likelihood of marriage because less time is available to search for a 
partner or socialize with a partner." Income may also affect marriage in distinct ways — by de- 
creasing financial pressure to get married or, perhaps, by increasing the financial security of a 
respondent who would otherwise have less family income. 

At the time of study entry all survey respondents were single mothers with children. How 
many of these mothers got married in the absence of the program? Figure 9.1 shows control 
group levels of marriage for the four survey sites using the sample of respondents who were in 
both the two-year and five-year surveys. This figure shows that across all sites between 8 percent 
and just over 20 percent of survey respondents reported being married at either the two-year or 
the five-year follow-up. This outcome was created to roughly capture a measure of “ever being 
married” during a five-year time period. This figure also shows that in Atlanta and Portland the 
majority of respondents in the control group who got married did so relatively late in the follow- 
up period, at some time during the last three years (that is, they were married at the five-year fol- 
low-up but not married at the two-year follow-up). 

Table 9.2 shows that none of these welfare-to-work programs had an impact on mar- 
riage. Although all programs showed a consistent pattern of increasing cohabitation, only the 
Riverside LFA program increased cohabitation significantly — by 4.6 percentage points, or 43 
percent. It may be the case that more program group members are engaging in less-formal rela- 
tionships than control group members, who are more likely to get married. Interestingly, in 
support of this hypothesis, program group members in Portland were 6.0 percentage points, or 
43 percent, more likely to cohabit, and 6.2 percentage points, or 26 percent, less likely to get 



'®An example of a policy that can be expected to affect marriage is streamlining eligibility rules for two-parent 
families on welfare by excluding any restrictions about the number of hours a spouse may work, often called the 
“100-hour” rule. 

"See Harknett and Gennetian, 2001, for detailed hypotheses about how welfare and employment programs, or 
changes in employment and income, may affect marital behavior. 
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Table 9.2 



Impacts on Marital Status During the Month Prior to the Five-Year Interview 



Site and Program 


Sanqjle 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 






Married, livine with spouse 




Atlanta Labor Force Attachment 


1,060 


9.8 


8.4 


1.3 


15.8 


Atlanta Human Capital Development 


1,135 


6.9 


8.4 


-1.5 


-18.2 


Grand Rapids Labor Force Attachment 


1,090 


22.8 


20.5 


2.3 


11.1 


Grand Rapids Human Capital Development 


1,102 


20.3 


20.5 


-0.2 


-1.0 


Riverside Labor Force Attachment 


1,213 


20.6 


22.0 


-1.4 


-6.5 


Lacked high school diploma or basic skills 


654 


18.6 


18.1 


0.5 


2.9 


Riverside Human Capital Development 


773 


21.8 


18.1 


3.7 


20.5 


Portland 


501 


17.4 


23.6 


-6.2 


-26.1 








Cohabiting 




Atlanta Labor Force Attachment 


1,060 


7.1 


6.7 


0.4 


5.8 


Atlanta Human Capital Development 


1,135 


6.7 


6.7 


0.1 


1.0 


Grand Rapids Labor Force Attachment 


1,090 


17.4 


15.9 


1.5 


9.4 


Grand Rapids Human Capital Development 


1,102 


16.2 


15.9 


0.3 


2.0 


Riverside Labor Force Attachment 


1,213 


15.3 


10.7 


4.6 ** 


42.6 


Lacked high school diploma or basic skills 


654 


15.9 


11.1 


4.9 * 


43.9 


Riverside Human Capital Development 


773 


13.7 


11.1 


2.6 


23.6 


Portland 


501 


19.9 


13.9 


6.0 


43.0 






Separated, divorced, or widowed 




Atlanta Labor Force Attachment 


1,060 


34.8 


37.2 


-2.4 


-6.5 


Atlanta Human Capital Development 


1,135 


36.1 


37.2 


-1.1 


-3.0 


Grand Rapids Labor Force Attachment 


1,090 


29.5 


33.7 


-4.1 ♦ 


-12.2 


Grand Rapids Human Capital Development 


1,102 


32.7 


33.7 


-0.9 


-2.8 


Riverside Labor Force Attachment 


1,213 


45.0 


45.8 


-0.8 


-1.8 


Lacked high school diploma or basic skills 


654 


44.1 


46.4 


-2.3 


-5.0 


Riverside Human Capital Development 


773 


42.1 


46.4 


-4.3 


-9.2 


Portland 


501 


38.8 


38.8 


0.0 


0.0 



(continued) 
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Table 9.2 (continued) 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 








Never married 




Atlanta Labor Force Attachment 


1,060 


48.4 


47.7 


0.7 


1.5 


Atlanta Human Capital Development 


1,135 


50.3 


47.7 


2.6 


5.4 


Grand Rapids Labor Force Attachment 


1,090 


30.3 


30.0 


0.4 


1.2 


Grand Rapids Human Capital Development 


1,102 


30.8 


30.0 


0.8 


2.7 


Riverside Labor Force Attachment 


1,213 


19.1 


21.4 


-2.3 


-10.9 


Lacked high school diploma or basic skills 


654 


21.3 


24.4 


-3.1 


-12.5 


Riverside Human Capital Development 


773 


22.3 


24.4 


-2.1 


-8.5 


Portland ^ 


501 


23.9 


23.7 


0.2 


0.7 



SOURCE: MDRC calculations trom the Five-Year Client Survey. 

NOTES: See Appendix A.2. 

Owing to missing values, sample sizes may vary. 
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married than control group members. These effects approached statistical significance (p = 
0.1 1). Thus, it appears that in Portland control group members were more likely to get married 
and program group members were more likely to cohabit. However, this hypothesized pattern 
of more program group members engaging in less-formal relationships than control group 
members is not clear in Atlanta and Grand Rapids. 

Only one other impact on marital status was found. Prograiri group members in the Grand 
Rapid LFA program were less likely to report being separated, divorced, or widowed by 4.1 per- 
centage points, or 12 percent. This latter effect is a result of either control group members mov- 
ing into marriage or cohabitation or program group members getting married and then separating 
or divorcing by the time of the five-year follow-up. The lack of more pervasive impacts on these 
point-in-time mieasures of marital status is not surprising for two reasons. As noted above, these 
programs were not intended to affect marital behavior. Also, point-in-time measures of marital 
status will not capture effects on marital status changes, and program effects on employment, for 
example, that occurred earlier in the follow-up period may have been more likely to affect the 
timing of marriage. Also, a respondent could have married after the two-year survey and divorced 
by the time of the five-year survey, and this change would not be measured in the survey. 

IV. Effects of Welfare-to-Work Approaches on Fertility 
and Household Composition 

A. Fertility 

While these welfare-to-work programs were not intended to alter fertility behavior, single 
mothers who are engaged in employment or training and who face a higher cost of becoming 
pregnant may be less likely to have another child. Or, because of age exemptions for the youngest 
child, they may have an incentive to have a baby in order to avoid any employment or training 
requirements. 

Table 9.3 (first panel) shows impacts on fertility. There were few significant impacts on 
the presence of a new baby in the household during the five-year follow-up period. Although 
there were no program impacts for program group members in the Riverside HCD program, LFA 
in-need sample members (a subgroup without a high school diploma or GED at study entry) were 
significantly more likely to have a new baby in the household, and, in fact, a similar effect that 
approaches statistical significance exists for all program group members in the Riverside LFA 
program (p = 0.12). (Because respondents were not specifically asked about whether or not they 
had a baby, this measure captures new births as well as new additions of an infant through mar- 
riage, adoption, or foster care.) 

B. Household Composition 

In the context of these programs, the immediate presence of other adults or family mem- 
bers may facilitate the transition to employment or may ease the burden of being employed dur- 
ing rotating or weekend hours because of available child care support. Thus, welfare-to-work 
programs may encourage single mothers to live with extended family. Decreases in income may 
also encourage program group members to live with extended family or a partner. Alternatively, 
increases in income may allow program group members to live independently, without the sup- 



National Evaluation of Welfare-to-Work Strategies 
Table 9.3 

Impacts on Fertility and Household Composition 

Sample Program Control Difference Percentage 
Site and Program Size Group (%) Group (%) (Impact) Change (%) 

Since random assignment 



Presence of a new baby 



Atlanta Labor Force Attachment 


1,069 


11.6 


12.4 


-0.8 


-6.7 


Atlanta Human Capital Development 


1,145 


12.5 


12.4 


0.1 


0.7 


Grand Rapids Labor Force Attachment 


1,097 


22.6 


21.7 


0.9 


4.3 


Grand Rapids Human Capital Development 


1,109 


22.2 


21.7 


0.5 


2.4 


Riverside Labor Force Attachment 


1,218 


25.5 


22.1 


3.4 


15.5 


Lacked high school diploma or basic skills 


656 


28.2 


23.1 


5.1 * 


22.2 


Riverside Human Capital Development 


111 


24.1 


23.1 


1.0 


4.5 


Portland 


504 


17.3 


22.7 


-5.3 


-23.6 


Durinc month prior to five-vear interview 






Lives alone 






Atlanta Labor Force Attachment 


1,071 


2.2 


5.8 


-3.6 *** 


-62.7 


Atlanta Human Capital Development 


1,146 


3.2 


5.8 


-2.6 ** 


-44.4 


Grand Rapids Labor Force Attachment 


1,097 


3.9 


3.5 


0.5 


13.1 


Grand Rapids Human Capital Development 


1,109 


4.1 


3.5 


0.6 


17.4 


Riverside Labor Force Attachment 


1,219 


3.7 


2.4 


1.3 


52.8 


Lacked high school diploma or basic skills 


657 


3.2 


1.2 


2.0 


159.9 


Riverside Human Capital Development 


778 


3.6 


1.2 


2.3 * 


190.4 


Portland 


504 


2.8 


3.2 


-0.4 


-11.6 






Lives with children onlv 




Atlanta Labor Force Attachment 


1,071 


62.9 


61.8 


1.1 


1.7 


Atlanta Human Capital Development 


1,146 


65.6 


61.8 


3.8 


6.1 


Grand Rapids Labor Force Attachment 


1,097 


45.1 


49.6 


-4.5 


-9.1 


Grand Rapids Human Capital Development 


1,109 


49.2 


49.6 


-0.3 


-0.7 


Riverside Labor Force Attachment 


1,219 


46.0 


46.6 


-0.6 


-1.2 


Lacked high school diploma or basic skills 


657 


50.3 


50.1 


0.1 


0.3 


Riverside Human Capital Development 


778 


47.2 


50.1 


-2.9 


-5.9 


Portland 


504 


48.6 


45.2 


3.5 


7.7 



(continued) 
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Table 9.3 (continued) 



Site and Program 


Sample Program Control Difference 

Size Group (%) Group (%) (Impact) 


Percentage 
Change (%) 




Lives with partner/spouse and children 




Atlanta Labor Force Attachment 


1,071 


14.4 


12.3 


2.1 




16.9 


Atlanta Human Capital Development 


1,146 


11.3 


12.3 


-1.0 




-8.3 


Grand Rapids Labor Force Attachment 


1,097 


34.4 


31.5 


2.9 




9.0 


Grand Rapids Human Capital Development 


1,109 


31.7 


31.5 


0.2 




0.6 


Riverside Labor Force Attachment 


1,219 


31.2 


28.5 


2.6 




9.3 


Lacked high school diploma or basic skills 


657 


29.4 


25.5 


3.9 




15.2 


Riverside Human Capital Development 


778 


29.9 


25.5 


4.4 




17.1 


Portland 


504 


28.5 


32.7 


-4.2 




-12.9 




Lives with adult extended family and any children 


Atlanta Labor Force Attachment 


1,071 


17.1 


16.7 


0.4 




2.7 


Atlanta Human Capital Development 


1,146 


18.2 


16.7 


1.5 




8.8 


Grand Rapids Labor Force Attachment 


1,097 


13.4 


12.5 


1.0 




7.9 


Grand Rapids Human Capital Development 


1,109 


9.8 


12.5 


-2.7 




-21.7 


Riverside Labor Force Attachment 


1,219 


14.3 


16.4 


-2.0 




-12.4 


Lacked high school diploma or basic skills 


657 


13.8 


17.1 


-3.3 




-19.3 


Riverside Human Capital Development 


778 


15.8 


17.1 


-1.3 




-7.9 


Portland 


504 


12.8 


12.7 


0.1 




0.4 






Liyes with an unrelated adult 






Atlanta Labor Force Attachment 


1,060 


10.0 


8.4 


1.6 




18.7 


Atlanta Human Capital Development 


1,135 


8.8 


8.4 


0.3 




3.7 


Grand Rapids Labor Force Attachment 


1,090 


20.8 


19.5 


1.3 




6.4 


Grand Rapids Human Capital Development 


1,102 


19.5 


19.5 


-0.1 




-0.3 


Riverside Labor Force Attachment 


1,213 


19.7 


17.5 


2.2 




12.5 


Lacked high school diploma or basic skills 


654 


18.9 


\73 


1.6 




9.0 


Riverside Human Capital Development 


774 


17.2 


\73 


-0.1 




-0.7 


Portland 


502 


25.9 


21.0 


4.9 




23.3 



SOURCE: MDRC calculations from the Five-Year Client Survey. 

NOTES: See Appendix A.2. 

Owing to missing values, sample sizes may vary. 
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port of relatives or other adults. Because it is often assumed that other household members con- 
tribute to family income, one outcome of particular policy interest is the proportion of single 
mothers who live with an unrelated adult. Unrelated adults who are not cohabiting partners may 
not contribute to family resources in the same way as spouses or cohabiting partners. Figure 9.2 
shows the proportion of control group members who lived with unrelated adults, for the sample 
of respondents who were in both the two-year and five-year surveys. Approximately 10 percent 
(in Atlanta) to 25 percent (in Riverside) of the control group members lived with an unrelated 
adult at the two-year follow-up or the five-year follow-up (again, a best estimate of “ever” occur- 
ring over a five-year follow-up period). These outcome levels are somewhat comparable to recent 
figures showing that 24 percent of single mothers in the poorest income decile lived with an un- 
related male in 1995 and 25 percent lived with an unrelated male in 1997.'^ 

The lower panels of Table 9.3 show impacts on living with an unrelated adult and various 
other measures of household composition. Overall, there were few program impacts on house- 
hold composition, except for impacts on living alone. The proportion of sample members who 
reported living alone at the five-year follow-up was less than 6 percent. The Atlanta LFA and 
HCD programs decreased the proportion of program group members who reported living alone 
by 3.6 and 2.6 percentage points, respectively; however, the Riverside HCD program increased 
the proportion by 2.3 percentage points. Although it is difficult to determine without further 
analysis, these effects may suggest that program group families in Atlanta are more likely to live 
with their children and program group families in the Riverside HCD program are less likely to 
live with their children. There is some, albeit rough, support for this hypothesis. As discussed in 
Chapter 11, very young children of program group members in Atlanta are less likely than chil- 
dren of control group members to have not lived with their mother because she could not care for 
them, and children of program group members in Riverside are more likely than children of con- 
trol group members to have not lived with their mother because she could not care for them. 

V. Effects of Welfare-to-Work Approaches on Housing Status 

Program effects on employment and income may also affect the likelihood of moving or 
seeking better or different housing. For example, a decrease in income may encourage or force 
program group members to move into less expensive housing, whereas an increase in income 
may allow them to move into better housing or own a home. 

Table 9.4 shows impacts on moving and housing. A high proportion of sample members 
— 66 to 86 percent of the control group — moved at least once during the five-year follow-up 
period. There was greater variation across sites in the proportion of control group members who 
moved more than once during the follow-up period: Nearly 30 percent moved more than once in 
Atlanta whereas up to 60 percent moved more than once in Riverside. Of the seven welfare-to- 
work programs, impacts on ever moving were produced only in Grand Rapids, where the LFA 
program increased the likelihood of moving by 7.6 percentage' points, or 9.8 percent, and the 
HCD program increased the likelihood of moving by 5.4 percentage points, or 7 percent. Al- 
though the Riverside LFA program did not increase the likelihood of moving once during the fol- 



'^Primus et al., 1999, using data from the Current Population Survey. 
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Figure 9.2 

Control Group Levels of Living with an Unrelated Adult 
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SOURCE: MDRC calculations from the Five-Year Client Survey. 



National Evaluation of Welfare-to-Work Strategies 
Table 9.4 

Impacts on Moving and Housing 

Sample Program Control Difference Percentage 
Site and Program Size Group (%) Group (%) (Impact) Change (%) 

Since random assignment 

Ever moved 



Atlanta Labor Force Attachment 


1,070 


67.0 


66.2 


0.8 


1.2 


Atlanta Human Capital Development 


1,145 


67.7 


66.2 


1.5 


2.3 


Grand Rapids Labor Force Attachment 


1,097 


85.6 


78.0 


7.6 *** 


9.8 


Grand Rapids Human Capital Development 


1,109 


83.4 


78.0 


5.4 ** 


6.9 


Riverside Labor Force Attachment 


1,219 


86.4 


84.0 


2.4 


2.8 


Lacked high school diploma or basic skills 


657 


83.6 


81.8 


1.8 


2.2 


Riverside Human Capital Development 


778 


81.8 


81.8 


0.0 


0.0 


Portland 


504 


84.8 


85.7 


-0.9 


-1.0 






Moved more than once 




Atlanta Labor Force Attachment 


1,068 


30.6 


29.7 


0.9 


3.1 


Atlanta Human Capital Development 


1,144 


31.1 


29.7 


1.4 


4.7 


Grand Rapids Labor Force Attachment 


1,096 


62.0 


53.9 


8.2 *** 


. 15.2 


Grand Rapids Human Capital Development 


1,108 


62.1 


53.9 


8.3 *** 


15.3 


Riverside Labor Force Attachment 


1,219 


67.5 


59.6 


-7 g ♦♦♦ 


13.3 


Lacked high school diploma or basic skills 


657 


63.4 


56.2 


7.1 ** 


12.6 


Riverside Human Capital Development 


778 


59.1 


56.2 


2.8 


5.0 


Portland 


504 


60.2 


59.4 


0.8 


1.3 


At five-vear interview 






Owns home 






Atlanta Labor Force Attachment 


1,062 


7.5 


5.3 


2.2 


42.3 


Atlanta Human Capital Development 


1,133 


6.6 


5.3 


1.3 


24.5 


Grand Rapids Labor Force Attachment 


1,080 


20.6 


18.2 


2.4 


13.3 


Grand Rapids Human Capital Development 


1,091 


22.3 


18.2 


4.1 * 


22.4 


Riverside Labor Force Attachment 


1,208 


10.7 


10.2 


0,5 


4.9 


Lacked high school diploma or basic skills 


650 


8.9 


9.1 


-0.2 


-2.5 


Riverside Human Capital Development 


766 


12.9 


9.1 


3.8 


41.8 


Portland 


495 


7.7 


9.4 


-1.6 


-17.3 
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Table 9.4 (continued) 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 






Lives in oublic/subsidized housine 




Atlanta Labor Force Attachment 


1,062 


49.1 


53.6 


-4.5 


-8.5 


Atlanta Human Capital Development 


1,133 


50.8 


53.6 


-2.9 


-5.3 


Grand Rapids Labor Force Attachment 


1,080 


20.1 


21.9 


-1.8 


-8.3 


Grand Rapids Human Capital Development 


1,091 


18.3 


21.9 


-3.6 


-16.5 


Riverside Labor Force Attachment 


1,208 


15.0 


14.6 


0.4 


2.6 


Lacked high school diploma or basic skills 


650 


18.1 


16.2 


1.9 


11.9 


Riverside Human Capital Development 


766 


17.6 


16.2 


1.4 


8.7 


Portland 


495 


29.2 


30.0 


-0.8 


-2.7 






Rents home or room 




Atlanta Labor Force Attachment 


1,062 


42.7 


40.6 


2.1 


5.1 


Atlanta Human Capital Development 


1,133 


40.9 


40.6 


0.3 


0.7 


Grand Rapids Labor Force Attachment 


1,080 


58.2 


58.1 


0.1 


0.1 


Grand Rapids Human Capital Development 


1,091 


57.1 


58.1 


-1.1 


-1.8 


Riverside Labor Force Attachment 


1,208 


72.4 


73.7 


-1.3 


-1.7 


Lacked high school diploma or basic skills 


650 


71.4 


73.2 


-1.8 


-2.4 


Riverside Human Capital Development 


766 


67.0 


73.2 


-6.2 ♦ 


-8.4 


Portland 


495 


59.8 


59.4 


0.4 


0.7 








Other housine 




Atlanta Labor Force Attachment 


1,062 


0.6 


0.4 


0.2 


50.4 


Atlanta Human Capital Development 


1,133 


1.7 


0.4 


1.3 ** 


302.5 


Grand Rapids Labor Force Attachment 


1,080 


1.1 


1.8 


-0.7 


-37.9 


Grand Rapids Human Capital Development 


1,091 


2.3 


1.8 


0.6 


33.7 


Riverside Labor Force Attachment 


1,208 


1.9 


1.5 


0.4 


25.1 


Lacked high school diploma or basic skills 


650 


1.6 


1.5 


0.1 


5.9 


Riverside Human Capital Development 


766 


2.4 


1.5 


0.9 


62.4 


Portland 


495 


3.3 


1.3 


2.0 


158.2 



SOURCE: MDRC calculations from the Five-Year Client Survey. 

NOTES: See Appendix A.2. 

Owing to missing values, sample sizes may vary. 
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low-up, it did increase the likelihood of moving more than once by 7.9 percentage points, or 13 
percent. A significant increase in moving more than once was also found for the Riverside in- 
need sample (respondents without a high school diploma or GED at study entry). 

Why did survey respondents move? For program group members (across sites) who 
moved, the most important reasons were that they felt that their current housing was old or bad, 
that is, the neighborhood was unsafe; they wanted to leave a bad relationship; or the space was 
too small. The next most important reason for moving was that respondents wanted and could 
afford a better place to live. And these reasons were more commonly cited by program group 
members who moved than by control group members who moved, which suggests that program 
impacts on moving can be interpreted as a positive outcome for respondents’ well-being. 

Table 9.4 also shows that there were few program effects on home ownership or other 
housing situations, such as renting or living in public or subsidized housing. One impact of note, 
given the impacts on moving previously discussed, was that the Grand Rapids HCD program in- 
creased the likelihood of owning a home at the time of the five-year follow-up by 4.1 percentage 
points, or 22 percent. It is striking that rates of home ownership among control group members 
are nearly twice as high in Grand Rapids as in all other sites. This can be partly attributed to rela- 
tively low housing costs and relatively little access to apartments or public/subsidized housing 
compared with the other sites. 

VI. Effects of Welfare-to-Work Approaches on Employment-Related 
and Domestic Abuse 

Current debates about welfare policy include whether or not and how to protect welfare re- 
cipients fi-om abusive relationships. Although these welfare-to-work programs did not test the ef- 
fects of time limits or special exemptions for victims of domestic abuse, increased employment and 
earnings may indirectly affect the incidence of employment-related or domestic abuse. On the one 
hand, increased employment may enhance self-esteem and encourage program group members to 
leave abusive relationships, or increased hours of employment may simply reduce contact with in- 
timate partners or family members who are abusive or remove individuals fi'om abusive situations. 
On the other hand, increased employment may exacerbate abuse that is occurring in a current rela- 
tionship as a partner or spouse negatively or violently reacts to the enhanced independence and self- 
sufficiency that accompanies employment.’^ In addition, because welfare benefits are replaced by 
earned income, increased employment in the context of these programs may not lead to increased 
personal resources or, possibly, the ability to leave an abusive relationship. 

As noted, information about job-related and domestic abuse was collected only for respon- 
dents in the Child Outcomes Study sample in Atlanta, Grand Rapids, and Riverside. Employment- 
related abuse is measured over a respondent’s lifetime. However, because information about the 
timing of domestic abuse is available, these outcomes are presented for two time periods — at any 
time during a respondent’s life and in the year prior to the five-year follow-up interview. 



'^Riger and Krieglstein, 2000. 
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Figure 9.3 shows the proportion of control group members who experienced domestic 
abuse at any time in their life (for example, had been threatened, yelled at, insulted, or physically 
harmed) and who experienced domestic abuse by an intimate partner in the year prior to the five- 
year follow-up interview. Nearly 50 to 70 percent of control group members reported ever being 
abused. Control group rates of any domestic abuse by intimate partners in the year prior to the 
five-year follow-up interview ranged from 15 to 20 percent. This is roughly 9 percentage points 
lower than the abuse rates documented for long-term recipients with young school-age children 
in the Minnesota Family Investment Program (MFIP) and for recipients and applicants in Flor- 
ida’s Family Transition Program, (FTP). However, state and national estimates suggest that ap- 
proximately 20 percent of the welfare population currently experiences domestic abuse and from 
40 to 70 percent experienced domestic abuse at any time during their life.’^ 

Impacts on employment-related abuse and domestic abuse that occurred at any time dur- 
ing a respondent’s life are shown in Table 9.5. Less that 10 percent of control group members 
reported experiencing any job harassment (being interrupted by phone or in person by someone) 
and up to approximately 30 percent reported experiencing job deterrence (being forced to quit or 
prevented from taking a job). The first three panels of Table 9.5 show that none of the programs 
in the three sites affected reports of any lifetime experience of having been discouraged from tak- 
ing a job, harassed while holding a job, or deterred from getting a job. The relatively low levels 
and lack of impacts on these outcomes are quite encouraging given recent literature that finds 
that conflicts with intimate partners or others, serve as an important barrier to work faced by for- 
mer welfare recipients.'^ Table 9.5 also shows reported any lifetime experience of domestic 
abuse. With one exception, none of programs affected reports of any lifetime experience of do- 
mestic abuse. Program group members in the Grand Rapids LFA program were 1 1 percentage 
points, or 17 percent, more likely to report any lifetime experience of domestic abuse. Further 
analyses suggest that this experience occurred before random assignment or during the first few 
years of follow-up and, for the most part, was nonphysical abuse by someone other than an inti- 
mate partner, such as a friend or family member (not shown). There were significant differences 
between the LFA and HCD program approaches on this outcome in Grand Rapids. 

Table 9.6 shows impacts on measures of domestic abuse in the year prior to the five-year 
follow-up interview. Although programs had few effects on measures of employment-related or 
domestic abuse that occurred at any point in a respondent’s life, the welfare-to-work programs 
did affect one important aspect of the quality of more recent relationships. In particular, in all of 
the LFA and HCD programs, program group members reported fewer incidences of experiencing 
physical domestic abuse (for example, hitting) during the year prior to the five-year follow-up 
interview, by 3 to 6 percentage points, than control group members, and nearly all of these pro- 
gram-control group differences achieved or approached statistical significance.'^ 



’‘*Gennetian and Miller, 2000; Bloom et al., 2000a. 

'^Allard et al., 1997; Raphael and Tolman, 1997. 

'^Danziger et al., 1999. 

*^The effect for the Atlanta HCD program is significant at the p = 0.12 level and the effect for the Riverside 
HCD program is significant at the p = 0. 1 1 level. 
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National Evaluation of Welfare-to-Work Strategies 
Figure 9.3 

Control Group Levels of Having Experienced Any Domestic Abuse 
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SOURCE: MDRC calculations from the Five-Year Client Survey. 



National Evaluation of Welfare-to-Work Strategies 
Table 9.5 



Impacts on Any Lifetime Experience of Employment-Related or Domestic Abuse 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 






Any lifetime experience of bavins been 








discouraged from taking a iob*‘ 




Atlanta Labor Force Attachment 


573 


29.1 


29.6 


-0.5 


-1.8 


Atlanta Human Capital Development 


646 


30.7 


29.6 


1.0 


3.5 


Grand Rapids Labor Force Attachment 


417 


36.1 


39.5 


-3.4 


-8.6 


Grand Rapids Human Capital Development 


398 


33.0 


39.5 


-6.5 


-16.4 


Riverside Labor Force Attachment 


502 


36.7 


36.5 


0.2 


0.6 


Lacked high school diploma or basic skills 


317 


31.6 


36.7 


-5.1 


-13.9 


Riverside Human Capital Development 


418 


34.5 


36.7 


-2.2 


-6.1 






Anv lifetime experience of bavins been 








harassed while holding a iob° 




Atlanta Labor Force Attachment 


565 


6.0 


8.1 


-2.1 


-25.6 


Atlanta Human Capital Development 


639 


6.2 


8.1 


-1.9 


-23.9 


Grand Rapids Labor Force Attachment 


415 


8.5 


9.9 


-1.4 


-14.1 


Grand Rapids Human Capital Development 


398 


8.0 


9.9 


-1.9 


-19.4 


Riverside Labor Force Attachment 


496 


9.9 


8.6 


1.3 


15.3 


Lacked high school diploma or basic skills 


311 


11.3 


7.2 


4.1 


56.7 


Riverside Human Capital Development 


413 


7.2 


7.2 


-0.1 


-0.8 


■— 




Anv lifetime experience of having been 








deterred from setting or holding a iob° 




Atlanta Labor Force Attachment 


575 


20.9 


20.8 


0.1 


0.4 


Atlanta Human Capital Development 


654 


20.2 


20.8 


-0.6 


-3.1 


Grand Rapids Labor Force Attachment 


417 


21.2 


29.0 


-7.8 * 


-26.8 


Grand Rapids Human Capital Development 


400 


23.4 


29.0 


-5.6 


-19.3 


Riverside Labor Force Attachment 


501 


27.0 


24.6 


2.3 


9.5 


Lacked high school diploma or basic skills 


315 


23.2 


23.7 


-0.5 


-2.1 


Riverside Human Capital Development 


415 


24.5 


23.7 


0.8 


3.6 



(continued) 
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Table 9.5 (continued) 



Sample Program Control Difference Percentage 

Site and Program Size Group (%) Group (%) (Impact) Change (%) 

Any lifetime experience of domestic abuse 



Atlanta Labor Force Attachment 


541 


51.5 


49.6 


1.9 


3.8 


Atlanta Human Capital Development 


610 


48.7 


49.6 


-1.0 


-1.9 


Grand Rapids Labor Force Attachment 


396 


75.8 


65.0 


10.8 ** 


16.6 


Grand Rapids Human Capital Development 


379 


62.8 


65.0 


-2.2 


-3.5 


Riverside Labor Force Attachment 


471 


61.4 


65.1 


-3.7 


-5.7 


Lacked high school diploma or basic skills 


289 


55.8 


61.2 


-5.4 


-8.8 


Riverside Human Capital Development 


380 


64.0 


61.2 


2.8 


4.6 



SOURCE: MDRC calculations from the Five-Year Client Survey. 

NOTES: See Appendix A.2. 

Owing to missing values, sanple sizes may vary. 

“Employment-related abuse, including job discouragement. Job harassment, and Job deterrence, is measured 
over a respondent’s lifetime. Some examples of employment-related abuse are as follows: someone's trying to 
discourage the respondent from finding a Job or going to work at any time in her life would be considered Job 
discouragement; the respondent's being harassed at her workplace over the telephone and/or in person at any 
time in her life would be considered Job harassment; someone's causing the respondent to quit or lose a Job at 
any time in her life would be considered Job deterrence. 
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National Evaluation of Welfare-to-Work Strategies 
Table 9.6 

Impacts on Experiences of Domestic Abuse in the Year 
Prior to the Five-Year Follow-Up Interview 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference Percentage 

(Impact) Change (%) 






Exoerienced anv abuse in prior year 




Atlanta Labor Force Attachment 


541 


21.4 


20.1 


1.3 


6.4 


Atlanta Human Capital Development 


610 


16.4 


20.1 


-3.7 


-18.3 


Grand Rapids Labor Force Attachment 


396 


24.5 


22.2 


2.3 


10.2 


Grand Rapids Human Capital Development 


379 


17.5 


22.2 


-4.7 


-21.2 


Riverside Labor Force Attachment 


471 


15.6 


19.2 


-3.5 


-18.4 


Lacked high school diploma or basic skills 


289 


16.0 


18.9 


-2.9 


-15.6 


Riverside Human Capital Development 


380 


19.2 


18.9 


0.3 


1.4 






Exoerienced ohvsical abuse in orior year 




Atlanta Labor Force Attachment 


541 


3.7 


7.4 


-3 7 ** 


-50.2 


Atlanta Human Capital Development 


610 


4.4 


7.4 


-2.9 


-39.7 


Grand Rapids Labor Force Attachment 


396 


10.9 


13.7 


-2.8 


-20.4 


Grand Rapids Human Capital Development 


379 


7.9 


13.7 


-5.8 ♦ 


-42.4 


Riverside Labor Force Attachment 


471 


7.5 


13.1 


-5.6 ♦♦ 


-42.5 


Lacked high school diploma or basic skills 


289 


6.0 


12.3 


-6.3 ♦♦ 


-51.4 


Riverside Human Capital Development 


380 


7.0 


12.3 


-5.2 


-42.6 






Experienced nonohvsical abuse in prior year 




Atlanta Labor Force Attachment 


541 


21.0 


19.7 


1.3 


6.6 


Atlanta Human Capital Development 


610 


15.8 


19.7 


-3.9 


-20.0 


Grand Rapids Labor Force Attachment 


396 


24.5 


21.7 


2.8 


13.0 


Grand Rapids Human Capital Development 


379 


17.5 


21.7 


-4.2 


-19.2 


Riverside Labor Force Attachment 


471 


15.6 


18.8 


-3.2 


-16.9 


Lacked high school diploma or basic skills 


289 


16.0 


18.4 


-2.4 


-13.0 


Riverside Human Capital Development 


380 


19.2 


18.4 


0.9 


4.7 



SOURCE: MDRC calculations from the Five-Year Client Survey. 

NOTES: See Appendix A.2. 

Owing to missing values, san^le sizes may vary. 

Physical abuse and nonphysical abuse are not mutually exclusive. 
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VII. Links Between E ffects on Household and Personal Circumstances 
and Particular Program Practices or Program Effects 
on Employment 

Of all of the outcomes and impacts examined in this chapter, only one may be differenti- 
ated by a particular program practice: The effect of the Grand Rapids LFA program on having 
experienced domestic abuse at any point during a respondent’s lifetime is significantly different 
from the effect of the HCD program. Although puzzling, it is unlikely that this effect depicts a 
general difference between LFA and HCD program approaches since these differences did not 
occur in Atlanta or Riverside, or on measures that were confined to the year prior to the five-year 
follow-up interview. These welfare-to-work programs had few effects on measures of any life- 
time experience of employment-related or domestic abuse. However, a striking and consistent 
pattern of impacts was found on the measure of quality of relationships in the prior year, with 
fewer program group members than control group members reporting experiences with physical 
domestic abuse (for example, by an intimate partner) in the year prior to the five-year follow-up 
interview. At the same time, there is no evidence that these programs altered respondents’ living 
arrangements at the time of the survey interview. 

The effects on physical domestic abuse are reassuringly positive and suggest that em- 
ployment can play a role in influencing some aspects of the quality of relationships. Unlike poli- 
cies that include an enhanced earned income disregard, increased employment, in the context of 
these welfare-to-work programs, did not always lead to enhanced personal resources or income 
that ultimately may help domestic abuse victims to leave their relationships. For example, recent 
MFIP findings suggest that MFIP’s financial incentives, rather than the added effects of MFIP’s 
participation requirements, produced the effects on decreased domestic abuse. Although these 
financial incentives contributed to increased employment, their most pronounced effects were to 
increase income and decrease poverty. The welfare-to-work programs evaluated in this report do 
not have a similar policy component, and certainly do not mirror MFIP’s effects on income. In 
fact, as previously mentioned, cumulative effects on average combined income varied considera- 
bly across sites. The differences in program effects on average combined income — that also ex- 
ist during the final year of follow-up — and similarities in program effects on domestic abuse 

across sites suggest that some other program effect may be contributing to decreased reported 
rates of physical abuse. 



Research suggests that women who have experienced recent abuse are as likely to be em- 
ployed as those who have not had a similar experience with abuse and that most women on wel- 
fare want to work and have work experience whether or not they have been victims of abuse. 
Program effects on employment somewhat support these research conclusions; that is, though 
rnost programs increased employment at some point during the follow-up period some programs 
did not necessarily increase employment during the last year of follow-up, yet these same pro- 
grams led to fewer reports of physical abuse. Thus, in some cases current employment may have 



'®Gennetian and Miller, 2000. 
’’Lyon, 2000. 
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played an important role, while in other cases early boosts in employment may have played an 
important role. Employment may not have always led to increased financial resources, but it may 
have increased self-esteem or self-efficacy, giving respondents the courage to leave abusive rela- 
tionships, or may have simply reduced contact with abusive partners or situations. 

Additional analyses were conducted to examine the relationship, if any, between effects 
on employment attributed to the LEA and HCD programs and effects on domestic abuse. These 
analyses do not reveal whether or not employment led to less physical abuse or vice versa, but 
can reveal whether or not effects on employment or program practices were at all related to ef- 
fects on physical abuse: Did the same program group members who experienced increased em- 
ployment also report less physical abuse? This hypothesis was examined by estimating impacts 
on joint outcomes depicting combinations of “being employed at any time during the follow-up 
period” and “experiencing physical domestic abuse in the year prior to the five-year follow-up 
interview” and “being employed in the year prior to the five-year follow-up interview” and “ex- 
periencing physical domestic abuse in the year prior to the five-year follow-up interview.” Ef- 
fects on these joint outcomes showed that both Riverside programs and the Grand Rapids HCD 
program significantly increased the likelihood of being employed and reporting no physical abuse 
in the year prior to the five-year follow-up interview. The pattern of effects was simile, though 
not statistically significant, for both Atlanta programs. Similar effects were not found for the 
joint measure of “employed and reporting no physical domestic abuse in the year prior to the 
five-year follow-up interview,” suggesting that early effects on employment may have contrib- 
uted to decreased reports of physical abuse later in the follow-up.^*^ 

Research suggests that abuse by intimate partners may be a critical bamer to employ- 
ment: Abusers sabotage women’s employment efforts, often leaving them with little choice ex- 
cept to quit or miss work or schooling.^' In this regard, it may have been the mandate to engage 
in employment-related activities and the very real threat of losing portions of the welfare benefit 
that helped women overcome any partner interference in their training or employment. 

Other possible explanations for observed effects on physical domestic abuse that occurred 
later in the follow-up period include program practices by caseworkers early in the follow-up pe- 
riod. Did program group members who reported less physical abuse later in the follow-up period 
also report that JOBS staff helped them with particular problems or provided them with particu- 
lar services early in the follow-up period? Program group members who reported physical abuse 
in the last year of follow-up were mOre likely than program group members who reported no 
abuse to also report at the two-year follow-up that JOBS staff were highly likely to help them 
with problems that made it difficult to participate in JOBS. Again, these differences were espe- 
cially pronounced among sample members in both Riverside program and the Grand Rapids 



“According to a similar analysis, the puzzling effects on ever experiencing abuse in the Grand Rapids LFA pro- 
gram do not appear to be linked with employment at all. 

^‘Riger, Ahrens, and Blickenstaff, 2001, Chapter 7. Some examples of sabotage include turning off alarm 
clocks, failing to fulfill child care responsibilities, destroying textbooks, and administering beatings so that a woman 
has highly visible bruises. Raphael, 1996. 

^^Unfortunately, the survey collected only information about whether or not a respondent “ever” experienced 
various barriers to employment by intimate partners and others, and, thus, these measures cannot be confined to the 
same period of time as the abuse measures. 
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HCD program. Thus, some of the impact on decreased physical abuse later in the follow-up pe- 
riod may be attributed to effects on “increased attention” by JOBS staff to deal with problems, 
such as domestic abuse, that might have made participation in employment or employment- 
related activities difficult. 

The TANF program of PRWORA paid special attention to women who experienced or 
were at imminent risk of serious partner violence and intimidation. As a result, states were given 
the option to engage in procedures to identify and assess risk, provide services, and, if necessary, 
temporarily waive program requirements for victims of abuse. The findings of this study are en- 
couraging because they highlight the fact that for some battered women, employment and/or 
caseworker attention to special services may lead to greater safety. Perhaps these women were 
able to. make changes in their relationships or separate from their abusers, or, through employ- 
ment, were able to physically remove themselves from abusive situations. Thus, the belief that 
most battered women need to be relieved of employment requirements is not bom out by this 
study. However, employment at a time of risk may not have such positive results for other abused 
women, particularly those who are at imminent risk and may require temporary relief from em- 
ployment requirements. Identifying these women and their needs is important so that various ap- 
proaches may be developed and implemented to best assist all battered women to move into em- 
ployment and off welfare.^^ The impacts on physical domestic abuse noted in this chapter are 
quite striking and merit further analysis and attention. Questions for further research include: 
How does employment and the characteristics of employment affect employment-related and 
domestic abuse? Does employment-related and domestic abuse affect employment stability? Are 

effects on employment-related and domestic abuse linked to effects on marital status or house- 
hold composition? 



The survey questions in the five-year interview in this study were not structured to identify women at immi- 
nent risk (or the effects of any exemptions for these women), but rather to capture whether or not personal or 
other relationships affected women’s ability to participate in program requirements or employment, and whether 

or not services by program staff, enhanced employment, or other resources, including income, can alter the qual- 
ity of relationships. 



er|c 
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Chapter 10 

Impacts on Child Care and Child Activities 

This chapter examines the impacts of seven of the NEWWS programs on child care use 
and, for a subgroup of school-age children (“focal” children), on activities during after-school 
hours from two perspectives: (1) as supporting mothers’ employment and transition from welfare 
and (2) as providing a context for children’s development. 

A primary goal of the NEWWS Evaluation was to consider the extent to which mothers as- 
signed to a welfare-to-work program moved from welfare to employment and to imderstand the 
supports that were important in bringing about such a transition. Child care was assumed to be of 
importance as a support for the transition to employment in that it provides for the supervision of 
children so that mothers can participate in work-related activities. Child care benefits were provided 
to both program and control group mothers who were participating in appropriate activities and 
were available while mothers were receiving welfare and for a year after they left welfare because 
of earnings.' Thus, a key question addressed in the first section of this chapter is whether there were 
program impacts on use of child care for purposes of employment and whether program group 
mothers made greater use of child care benefits than control group mothers. 

National survey data indicate that child care arrangements are reported by low-income 
mothers who are not employed as well as by mothers who are employed (albeit at a lower rate). 
Nonemployed mothers appear to use child care in order to expose their children to stimulating 
early childhood education environments and to support their own participation in education and 
training activities.^ Within the context of the JOBS welfare-to-work programs studied here, the 
possibility exists that children’s exposure to child care in the program groups might increase not 
only because of employment, but also as mothers increase their participation in education or 
training activities and perhaps come to value exposure to educational experiences and want these 
experiences for their children as well as for themselves. 

The second section of this chapter explores the extent and nature of children’s exposure 
to nonmatemal care for any purpose, not only as a support for employment. The underlying ques- 
tion is whether mothers’ assignment to one of the welfare-to-work programs results in changes in 
the amount and type of nonmatemal care that children experience. The section that explores child 
care as a support for employment takes into accoimt child care for all the children in the family 
under age 13 while the section that explores child care for any purpose focuses on the focal chil- 
dren in the Child Outcomes Study, who were aged 8 to 10 years at the five-year follow-up. This 



'Child care benefits may have changed over time, particularly after the 1996 welfare reform and consolidation of 
child care binding into the Child Care and Development Fund. A thorough review of these changes by site is beyond 
the scope of this chapter. Moreover, many sample members may not have remained eligible for child care assistance 
after 1996 since they were off welfare and had been off for some time. 

^Tout et al., 2001. 

^Tout et al. document differences by income level in reasons for child care use by nonemployed mothers, with 
employment preparation activities playing a larger role for lower-income families. 
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section presents impacts on both what mothers describe as regular child care arrangements for this 
school-age child and on supervision and activities during the after-school hours. 

I. Key Findings 

• In general, there were very few program impacts on child care at the time 
of the survey. This is perhaps not surprising, given that there was no child 
care component specific to the JOBS program and that the impacts of these 
programs on employment tended to decline by the end of the follow-up period. 

• Most programs increased mothers’ use of child care after leaving welfare 
because of earnings across the full follow-up period. However, only three 
of seven programs increased use of transitional child care benefits over 
this period. Impacts on transitional benefits may be more limited owing to 
site variation in the take-up rate of this benefit. 

• Child care impacts for a recent period occurred only in the few programs 
with sustained employment impacts. Portland’s program had an impact on 
child care use during a recent spell of employment. In addition, both of River- 
side’s programs changed time use “on a recent weekday,” reducing time spent 
with the mother and increasing time with another adult. At the same time, in 
the Riverside LFA program, a smaller proportion of program group families 
than control group families used formal child care as their regular main ar- 
rangement. 

• There were no impacts on the extent of participation in what mothers de- 
scribed as regular child care arrangements for their children aged 8 to 10 
at the time of the final follow-up. None of the programs affected total num- 
ber of hours in child care, use of a regular arrangement, or use of multiple ar- 
rangements. 

• Self-care among children aged 8 to 10 generally did not increase when 
mothers had been assigned to a welfare-to-work program, and there were 
no program impacts on focal children’s participation in an after-school 
activity or lesson. Although there were no impacts on self-care “in the past 
week, the proportion of control group children in self-care ranged from 9 
percent in Riverside to 12 percent in Grand Rapids. For the subgroup of moth- 
ers with limited education, the Riverside LFA program increased the propor- 
tion of children who spent any time “on a recent weekday” unsupervised by 
adults The programs examined here did not increase (or decrease) children’s 
exposure to potentially beneficial after-school activities and lessons.'* 



While some previous research suggests that participation in an organized after-school program can have favor- 
able implications for the development of children in low-income families (Bos et al., 1999), these implications are 
likely to vary also by the quality of after-school care. The quality of child care arrangements has equally important 
implications for preschool-age children. 



II. Analysis Issues 

A. Mother’s Employment and Child Care Use 

The child care questions asked of the client survey sample (families in four research sites) 
provide the basis for the discussion below of child care use as a support for employment. For the 
full follow-up period, respondents were asked about the use of child care after leaving welfare 
because of earnings, and about receipt of transitional child care benefits.^ Unlike the survey in- 
formation that was collected at the two-year follow-up point, the five-year survey did not ask 
about receipt of child care subsidies other than transitional child care benefits. For the more lim- 
ited time period of the “most recent spell of employment,” respondents were asked about use of, 
and out-of-pocket expenses for, child care while employed.^ 

Children of client survey sample members were between ages 1 and 18 at random as- 
signment and between ages 6 and 23 at the time of the five-year follow-up. The questions about 
child care as a support for employment generally were asked about all children in the family who 
were under age 13 at the time when the care was used. This could have included child care for 
children who joined the family after random assignment. ’ These analyses were performed with 
the family as the unit of analysis, because the child care information could not be linked to spe- 
cific children in each family. Although it is possible to conduct analyses by the age of the young- 
est child in the household to capture roughly the different needs and circumstances for children 
across the age range, especially for those with toddlers and preschoolers rather than only school- 
age children, this type of analysis was beyond the scope of the chapter. 

It is critical to emphasize that child care benefits were available to both program and con- 
trol group mothers based on their engagement in work-related activities or their employment fol- 
lowing an exit from welfare. That is, there was no distinctive child care treatment specific to the 
program groups within the NEWWS Evaluation (such as provision of more extensive subsidies, 
special help when there were problems with child care arrangements, or access to particular high- 
quality child care). Accordingly, any impacts on child care participation and use of child care 
benefits for purposes of employment are related to program impacts on mothers’ work-related 
activities or to other program features, such as the programs’ mandatory nature, that might have 



^All impacts on leaving welfare because of earnings, on using child care after leaving welfare because of earn- 
ings, and on using transitional child care benefits are experimental. That is, all sample members are included in the 
analyses. 

®For ease of presentation, throughout this chapter child care during the most recent employment spell will simply 
be referred to as child care while employed. 

’There are 13,726 children in the Five-Year Client Survey sample: the sample in four sites (Atlanta, Grand Rap- 
ids, Riverside, and Portland) for whom there are interviews as well as administrative data (see Chapter 2). Of these 
children, 6 percent were aged 2 or under at baseline (about 5 to 7 at follow-up), 25 percent were aged 3 to 5 at base- 
line (about 8 to 10 at follow-up), 22 percent were aged 6 to 9 at baseline (about 11 to 14 at follow-up), and 33 per- 
cent were aged 10 to 18 at baseline (about 15 to 23 at follow-up). Approximately 1 1 percent were bom into the fami- 
lies participating in the client survey sample between baseline and the five-year follow-up and another 2 percent did 
not have their birthdays reported. For more information about the percentage of families containing children bom 
after random assigiunent, see Chapter 9 of this report. Those who did not have their birthdays reported were included 
in the all-child analyses but excluded from analyses focusing on age subgroups. The child care measures linking 
child care and employment focus on children who were under age 13 at the time that the child care was used. 



led directly to increased employment. Thus, it is important to ask whether and to what extent 
families made greater use of child care and of child care benefits for purposes of employment 
when they had been assigned to a JOBS welfare-to-work program. 

Findings presented in Chapter 4 of this report regarding program impacts on employment 
underscore the importance of distinguishing between measures for the full follow-up period and 
measures for a recent and more limited time period. While impacts on average number of quar- 
ters employed and on total earnings were widespread across the full follow-up period, impacts 
were more circumscribed for measures of employment and earnings at the time of the five-year 
follow-up, and in a number of sites differences were no longer statistically significant at this 
point. As noted in Chapter 4, the narrowing of differences by the five-year point between the 
program and control groups on employment outcomes reflects, in part, the fact that control group 
mothers increasingly sought employment on their o\vn over the follow-up period and the fact that 
some program group members left employment over time. In addition, in Atlanta and Grand Rap- 
ids some or all control group families became subject to new state welfare-to-work initiatives in 
the final years of the evaluation, following passage of the 1996 welfare law.^ Thus, in looking at 
program impacts on child care as a support for employment and the transition fi-om welfare, it 
seems reasonable to predict more widespread impacts on measures looking across the full follow- 
up period than on measures pertaining to the recent spell of employment. 

It will also be important to keep in mind the ages of the children in the evaluation sample. 
While some mothers bore children over the course of the five-year follow-up, children who had 
already been bom at the start of the follow-up period increasingly moved into the school-age and 
adolescent age groups as the follow-up period proceeded. Findings fi"om recent national surveys 
indicate that while child care use increases with child age during the preschool years (that is, for 
children aged 5 or under but not yet in kindergarten),^ child care use begins to decline once chil- 
dren enter school, though substantial numbers of school-age children continue to be in regular 
child care arrangements.'® Thus, by the time of the final follow-up, a higher proportion of the 
children in the NEWWS Evaluation might be in an age range for which employed mothers no 
longer report regular child care arrangements for their children. Against this backdrop of dimin- 
ished overall child care use, program impacts might also be less likely to occur. 

In sum, across the full five-year follow-up as well as at the end of the follow-up, it is im- 
portant to consider the extent to which child care played a role in supporting mothers’ employ- 
ment and departure fi-om welfare receipt. Yet because the child care treatment did not differ for 
pro^am and control groups, because the pattern of employment impacts narrowed in a number 
of sites over the course of the follow-up, and because of the increasing ages of children in the 
evaluation sample, the possibility exists that impacts will not be strong or widespread, particu- 
larly when focusing on child care use during mothers’ most recent spell of employment. 



*See the discussion of control group exposure to new policies in Chapter 1 . 

’Toutet al., 2001. 

'“For example, findings from the 1997 National Survey of American Families indicate that 55 percent of chil- 
dren aged 6 to 9 and 35 percent of those aged 10 to 12 with employed mothers were in a supervised nonparental 
child care arrangement (Capizzano, Tout, and Adams, 2000). 
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B. Children’s Experiences in Child Care and Activities 



The child care questions included in the more extensive interview for the Child Out- 
comes Study sample (families in three of the sites) provide the basis for the discussion below 
of children’s recent experiences in child care whether used as a support for employment or not. 
These questions focus specifically on the focal child rather than on all children in the family, 
and they focus exclusively on the time period of the final follow-up. Focal children were be- 
tween about ages 8 and 10 at the time of the five-year follow-up. Regarding child care “in the 
past week,” respondents were asked about use of any self-care, whether there was a regular 
child care arrangement for the focal child, type of regular arrangement used, whether multiple 
arrangements were used, hours in child care across all regular arrangements, and reliance on a 
caregiver aged 17 or under. To address the possibility that mothers may not describe the 
supervision they use for their school-age children as child care, mothers were also asked about 
whom the focal child was with in half-hour time periods from 3 P.M. to midnight “on a recent 
weekday.” To address the possibility of program impacts on participation in activities or les- 
sons, mothers’ reports of participation in such activities on the recent weekday afternoon and 
evening was also analyzed. Self care was a category in the questions both about child care in 
the past week and about supervision on the recent weekday afternoon and evening. While 
measures about a recent afternoon and evening provide useful information, such measures 
likely underestimate activities that occur more sporadically. 

A substantial body of research indicates that children’s experiences in child care are re- 
lated to their development." Much of this research has focused on variations in child care 
quality and in children’s development and confirms that across both formal and informal types 
of child care, higher-quality care is associated with more advanced cognitive development and 
more positive social behavior.'^ In addition, however, there is also emerging evidence that type 
of child care is related to development. While quality varies widely within both formal and in- 
formal types of child care, recent findings suggest that beyond variation in quality, participa- 
tion in more formal, center-based programs is related to children’s cognitive school- 
readiness. This may be, for example, because of the greater likelihood that a formal curricu- 
lum is used in center-based programs or that there is a greater availability of books and other 
cognitively stimulating materials in such settings. 

In keeping with this finding from the larger research literature, earlier analyses of chil- 
dren’s development in the NEWWS Child Outcomes Study have linked participation in formal 
child care settings with better concurrent developmental outcomes (higher scores on assessments 
of cognitive school-readiness).''* Accordingly, in the analyses presented here, a key issue will be 
not only whether mothers’ assignment to a JOBS welfare-to-work program affected the overall 



"National Research Council and Institute of Medicine, 2000. 

'^National Research Council and Institute of Medicine, 2000; Vandell and Wolfe, 2000 

"NICHD Early Child Care Research Network, 2000; see also U.S. Department of Education, National Center 
for Education Statistics, 1995. The NICHD findings suggest that extensive participation in formal child care ar- 
rangements may also simultaneously increase children’s externalizing behavior problems. 

"Zaslow et al., 1998; Zaslow et al., 1999b. 
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extent of children’s exposure to nonmatemal care, but also whether there were shifts in type of 
care, especially between formal and informal types. 

Some concern has been expressed that mandatory participation in a welfare-to-work 
program might push some mothers to leave school-age children to care for themselves while 
the mothers engage in work or work-preparation activities or that children might be left in the 
care of young caregivers (for example, older siblings). National data indicate that self-care as a 
primary child care arrangement increases during the early school years, from about 5 percent 
for children aged 6 to 9 to about 24 percent for those aged 10 to 12 with employed mothers.'^ 
While findings linking self-care with children’s development appear to differ by age and 
characteristics of the child, there is some evidence that regularly spending time in unstructured 
and unsupervised settings is deleterious for children’s development. Given these concerns, 
this chapter also examines whether there were program impacts on families’ use of self-care 
and care by a young caregiver. 

At the time of the five-year follow-up, the focal children studied were aged 8 to 10. As 
noted above, in this age range a decreasing proportion of mothers report child care use. The pos- 
sibility exists, however, that mothers are nevertheless making arrangements for the supervision 
of their school-age children, but not describing these arrangements as child care. To capture a 
range of possible supervisory situations that school-age children might be in, outcomes examined 
include not only child care arrangements that the focal children participated in “in the past 
week,” but also a broader description of supervision of the children “on a recent weekday” (spe- 
cifically whether they were with the mother, with another adult, with only a young peer or sib- 
ling, or in self-care). 

After-school activities (such as participation in clubs, sports, or lessons) might also be 
providing supervision in the mother’s absence and yet not be labeled child care. Accordingly, this 
section also includes after-school activities during a recent weekday afternoon. Indeed, in a re- 
cent evaluation of a New Hope work incentive and support program, program impacts on use of 
after-school child care and participation in activities were found, especially for boys. This pro- 
gram also had favorable impacts on the social and academic outcomes of school-age boys. The 
researchers hypothesize that participation in after-school care and activities might have protected 
the school-age boys in the study from exposure to street activity in dangerous neighborhoods and 
exposed them instead to positive, sometimes academically oriented, supervised activities.'^ 



*^In the analyses presented here, formal care included care in a day care center, before- or after-school care 
sponsored by a school or church, summer camp, a boysVgirls’ club, YMCAA^WCA, or a lesson/activity. Informal 
care included care by the mother’s spouse/partner, a relative, or a neighbor. 

*^Type of care is more appropriately studied through interview measures than quality of care, which is best stud- 
ied through direct observation. The interview methodology of the NEWWS Evaluation thus better addresses the is- 
sue of type of care than quality of care. The lack of direct observation of child care quality is a clear limitation in this 
chapter. 

*’Capizzano, Tout, and Adams, 2000. 

*^Capizzano, Tout, and Adams, 2000. 

*^Bos et al., 1999 



III. Child Care Use as a Support for Employment 

This section describes findings on the use of child care as a support for employment and 
on the use of transitional child care benefits. Findings on control group levels are presented first, 
followed by findings on program impacts. 

A. Control Group Levels of Child Care 

Across the full follow-up period, the use of child care arrangements after leaving welfare 
because of earnings varied considerably for control group members in the client survey sample 
across the sites. As Table 10.1 shows, control group members in Atlanta and Riverside reported 
the lowest levels of child care use (17 to 24 percent), whereas control group members in Portland 
reported the highest level (44 percent). Receipt of transitional child care benefits also varied con- 
siderably among control group members across the sites, ranging from approximately 5 percent 
in Riverside to 31 percent in Portland. Among those control group meimbers who used child care 
after leaving welfare because of earnings, receipt of transitional child care benefits ranged from 
20 percent in Riverside to 70 percent in Portland. 

Approximately one-third to one-half of control group members in the client survey sam- 
ple reported using child care while employed, and one-fifth to one-third of control group mem- 
bers in this sample reported paying for child care out-of-pocket. Thus, of those control group 
members who used child care while employed, approximately 60 percent paid out-of-pocket for 
at least some of this child care (the proportion of those who paid out-of-pocket for care divided 
by the proportion of those who reported using child care while employed). 

The wide cross-site variation in take-up of transitional child care benefits may be related 
to a number of things including whether, or to what extent, families were informed of these bene- 
fits and/or encouraged to use them, bureaucratic hurdles in applying for and maintaining benefits, 
variation in preferences or need for child care, and/or variation in coverage of certain types of 
care. Yet a majonty of control group families who were using child care for purposes of em- 
ployment had out-of- pocket expenses for care. These findings point to the potential importance 
of active outreach in informing families of the child care benefits for which they are eligible and 
suggest that child care expenses may play a role in overall family income for many families, 
counteracting some of the benefits of earnings. 

B. Impacts on Child Care Use as a Support for Employment 

Findings that pertain to the full five-year follow-up show that most of the programs (five 
of the seven in the client survey sample) increased the likelihood of mothers’ child care use after 
leaving welfare because of earnings. All of the programs except Grand Rapids HCD and Portland 
increased child care use in this way.^' 

A smaller number of programs significantly increased receipt of transitional child care 
benefits, frnpacts were significant for both Atlanta programs and for the Riverside LFA program 



^®See Chapter 1 for a discussion of these differences. 

^'Xhe impact in the Grand Rapids HCD program on using child care after leaving welfare for earnings was sig- 
nificant at the p = . 19 level. 



National Evaluation of Welfare-to-Work Strategies 
Table 10.1 

Impacts on Child Care Use 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Percentage 
Change (%) p- 


-Value 




Used child care after leaving AFDC because of earnings (Vo) 




Atlanta Labor Force Attachment 


1,054 


35.8 


24.3 




47.4 


0.00 


Atlanta Human Capital Development 


1,123 


29.9 


24.3 


5.7 ** 


23.4 


0.02 


Grand Rapids Labor Force Attachment 


1,084 


46.0 


38.1 


-j ^ *** 


20.7 


0.00 


Grand Rapids Human Capital Development 


1,098 


41.7 


38.1 


3.7 


9.6 


0.19 


Riverside Labor Force Attachment 


1,188 


31.3 


24.5 


6.7 *** 


27.5 


0.01 


Lacked high school diploma or basic skills 


635 


23.0 


16.8 


6.2 ** 


37.1 


0.05 


Riverside Human Capital Development 


750 


24.1 


16.8 


■7 3 ♦♦ 


43.4 


0.02 


Portland 


494 


47.7 


44.2 


3.5 


7.8 


0.45 






Received transitional child care benefits (%) 




Atlanta Labor Force Attachment 


1,053 


19.2 


9.2 


10.0 *** 


109.0 


0.00 


Atlanta Human Capital Development 


1,120 


13.0 


9.2 


3.9 * 


42.1 


0.05 


Grand Rapids Labor Force Attachment 


1,078 


15.0 


11.7 


3.3 


27.8 


0.11 


Grand Rapids Human Capital Development 


1,089 


14.7 


11.7 


3.0 


25.2 


0.15 


Riverside Labor Force Attachment 


1,182 


7.8 


4.9 


3.0 ** 


60.3 


0.03 


Lacked high school diploma or basic skills 


632 


6.5 


3.0 


3.4 ** 


112.7 


0.03 


Riverside Human Capital Development 


748 


3.4 


3.0 


0.3 


11.0 


0.84 


Portland 


492 


34.8 


31.0 


3.8 


12.2 


0.39 




Used transitional child care benefits amons those who used 






child care after leaving welfare because of earnings (%) 




Atlanta Labor Force Attachment 




53.6 


37.8 


15.8 


41.8 




Atlanta Human Capital Development 




43.6 


37.8 


5.8 


15.2 




Grand Rapids Labor Force Attachment 




32.6 


30.7 


1.8 


5.9 




Grand Rapids Human Capital Development 




35.1 


30.7 


4.4 


14.3 




Riverside Labor Force Attachment 




25.1 


20.0 


5.1 


25.8 




Lacked high school diploma or basic skills 




28.1 


18.1 


10.0 


55.1 




Riverside Human Capital Development 




14.0 


18.1 


-4.1 


-22.6 




Portland 




72.9 


70.0 


2.9 


4.1 





(continued) 
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Table 10.1 (continued) 



Sample Program Control Difference Percentage 

Site and Program Size Group Group (Impact) Change (%) p-Value 



Atlanta Labor Force Attachment 
Atlanta Human Capital Development 


1,066 

1,140 


Grand Rapids Labor Force Attachment 
Grand Rapids Human Capital Development 


1,093 

1,097 


Riverside Labor Force Attachment 
Lacked high school diploma or basic skills 
Riverside Human Capital Development 


1,212 

653 

774 


Portland 


498 


Atlanta Labor Force Attachment 
Atlanta Human Capital Development 


1,061 

1,140 


Grand Rapids Labor Force Attachment 
Grand Rapids Human Capital Development 


1,087 

1,093 


Riverside Labor Force Attachment 
Lacked high school diploma or basic skills 
Riverside Human Capital Development 


1,210 

653 

774 


Portland 


496 


Atlanta Labor Force Attachment 
Atlanta Human Capital Development 


1,061 

1,140 


Grand Rapids Labor Force Attachment 
Grand Rapids Human Capital Development 


1,087 

1,093 


Riverside Labor Force Attachment 
Lacked high school diploma or basic skills 
Riverside Human Capital Development 


1,210 

653 

774 


Portland 


496 



Used child care while emoloved (%) 






33.2 


33.2 


0.1 


0.2 


0.97 


32.1 


33.2 


-1.1 


-3.3 


0.67 


49.2 


45.3 


4.0 


8.8 


0.12 


43.8 


45.3 


-1.5 


-3.3 


0.57 


42.9 


42.4 


0.5 


1.2 


0.84 


43.9 


39.3 


4.6 


11.8 


0.19 


40.6 


39.3 


1.3 


3.3 


0.72 


50.7 


43.8 


6.9 ♦ 


15.8 


0.09 


Paid 


for child care out-of-pocket (Vo) 






18.5 


19.4 


-0.9 


-4.6 


0.69 


20.2 


19.4 


0.8 


4.1 


0.72 


29.9 


29.5 


0.4 


1.2 


0.89 


28.5 


29.5 


-1.0 


-3.4 


0.70 


26.4 


26.8 


-0.4 


-1.6 


0.86 


26.7 


22.2 


4.5 


20.4 


0.16 


24.1 


22.2 


1.9 


8.7 


0.55 


32.3 


28.1 


4.3 


15.2 


0.32 



Total monthly out-of-pocket cost of child care 



29.5 


32.1 


-2.6 


-8.2 


0.59 


34.6 


32.1 


2.4 


7.5 


0.62 


70.3 


68.7 


1.5 


2.2 


0.85 


65.5 


68.7 


-3.2 


-4.7 


0.68 


62.0 


61.8 


0.2 


0.4 


0.97 


59.1 


48.6 


10.5 


21.5 


0.28 


61.5 


48.6 


12.9 


26.4 


0.19 


113.5 


62.1 


51.4 ** 


82.8 


0.02 



SOURCE: Child Trends calculations from the Five-Year Client Survey. 
NOTES: See Appendix A.2. 



for both the full and in-need sample. Atlanta LFA program group members were 11.5 percent- 
age points more likely to have used child care after leaving welfare and were 10 percentage 
points more likely to have received transitional child care benefits than control group members. 
In addition, HCD program group members in Atlanta were 5.7 percentage points more likely to 
have used child care after leaving welfare and 3.9 percentage points more likely to have received 
transitional child care benefits than control group members. These impacts suggest that much of 
the increased child care use after leaving welfare because of Atlanta’s welfare-to-work program 
was covered by transitional child care benefits. In contrast, though Riverside HCD program 
group members were 7.3 percentage points more likely than control group members to use child 
care after leaving welfare because of earnings, there were no significant differences in receipt of 
transitional child care benefits. 

Across the full follow-up period, then, most programs increased reliance on child care af- 
ter leaving welfare because of earnings, but fewer than half increased receipt of transitional child 
care benefits. It has been noted that there was substantial cross-site variation in control group 
families’ receipt of transitional child care benefits. In general, the findings suggest the need for 
expanding information about subsidies, altering eligibility or increasing funding, and simplifying 
the administration of subsidies to encourage families to use transitional child care benefits. 

Impact findings on child care use in connection with mothers’ recent spell of employment 
are yet more limited. None of the LFA or HCD programs increased child care use while em- 
ployed in a recent spell (but see the findings presented below for the COS sample on sustained 
employment for mothers and focal children’s time use in both programs in Riverside). Portland 
program group members, on the other hand, were 6.9 percentage points, or 16 percent, more 
likely than control group members to use child care while employed in a recent spell. While none 
of the LFA or HCD programs significantly increased the proportion of families with any out-of- 
pocket expenses for child care, program group members in Portland had significantly higher total 
monthly out-of-pocket child care costs than control group members. Portland program group 
members who used child care while employed paid approximately $82 more in total monthly out- 

90 

of-pocket costs than control group members who used child care while employed. 

In general, the findings for child care as a support for employment are in keeping with the 
prediction of more extensive impacts for the full follow-up period — for which employment im- 
pacts were more widespread — than for a recent spell of employment — when employment im- 
pacts were more limited. While there is a fairly widespread pattern of impacts on child care use 
while transitioning from welfare across the full follow-up, there is very little evidence of impacts 
on child care use for a recent spell of employment. The impacts that did occur were concentrated 
in Portland (and in Riverside, as discussed below), where there were impacts on sustained or re- 



^^The impact on receipt of transitional child care benefits is significant at the p = 0. 1 1 level for the Grand Rapids 
LFA program and at the p = 0. 1 5 level for the HCD program. 

^^The total monthly out-of-pocket cost of child care was $225 ($114 divided by 0.507, or the proportion of the 
sample who used child care while employed) for program group members and $142 for control group members. This 
is a nonexperimental comparison. 





cent employment. In Portland, there were impacts on full-time employment and employment in 
the last quarter of year 5 for families with yoimg children.^"* 

IV. Child Care Use as a Context for Development 

This section describes the extent and type of child care experienced by young school- 
age children in the Child Outcomes Study sample, whether used as a support for eniployment 
or not. It focuses on child care arrangements for the focal children, on how they spend time 
and their participation in activities. It should be kept in mind that all of the focal children were 
aged 8 to 10 and that all measures pertain to a very recent time period, either the past week or a 
recent weekday. 

A. Control Group Levels of Child Care - 

Table 10.2 shows that between about 34 and 43 percent of focal children in the Child 
Outcomes Study control groups were reported by mothers to be in a regular child care ar- 
rangement “in the past week,” that is, in the week prior to the five-year follow-up. The main 
arrangement was much more likely to be informal, such as care by a relative or neighbor in the 
child’s home or another home (used by 24 to 36 percent of control group families, depending 
on site), than formal, such as care provided by a day care center or provided after school by the 
school (used by 8 to 19 percent). Children who were in a regular arrangement spent 20 to 26 
hours per week on average in child care (that is, about 4 to 5 hours per day if care occurred 
five days a week). Only 6 to 10 percent of families reported relying on multiple arrangements. 
Only a small proportion of families reported relying on a provider aged 1 7 or under in the main 
arrangement for the focal child (about 3 to 5 percent). Use of self-care in the past week was 
reported for about 9 to 12 percent of control group families, which falls within the range re- 
ported for national samples. 

In sum, findings for control group families indicate use of child care by a minority, but a 
large one, of families even when the focus is limited to school-age children. When care was used, 
it involved a substantial portion of the children’s day. These findings suggest that attention to the 
nature of children’s child care arrangements should not cease with low-income children’s entry 
into school. 

B. Impacts on Child Care Use 

1. Child Care for the Focal Child 

There is no evidence on measures of child care use for the focal child in the past week 
that assignment to one of the LFA or HCD programs affected extent of reliance on child care ar- 
rangements at the time of the five-year follow-up. There were no impacts on families’ use of a 
regularly scheduled child care arrangement or on sporadic use of child care in the past week. To- 



Although there were no impacts in the aggregate in Portland, there were increases in employment in the last 
quarter of year 5 and in full-time employment for families who had preschool -age and young school-age children at 
baseline. 

^’See Capizzano, Tout, and Adams, 2000. 
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National Evaluation of Welfare-to-Work Strategies 
Table 10.2 

Impacts on Child Care for the Focal Child 



Site and Program 


Sample 

Size 


Program 

Group 


Control Difference 
Group (Impact) 


Percentage 
Change (%) 






Used a reeular child care arraneement 








in the past week (Vo) 




Atlanta Labor Force Attachment 


580 


41.5 


39.7 


1.8 


4.6 


Atlanta Human Capital Development 


650 


37.8 


39.4 


-1.6 


-4.1 


Grand Rapids Labor Force Attachment 


394 


41.9 


43.4 


-1.5 


-3.4 


Grand Rapids Human Capital Development 


380 


36.9 


41.8 


-4.9 


‘-11.6 


Riverside Labor Force Attachment 


493 


33.9 


33.5 


0.4 


1.3 


Lacked high school diploma or basic skills 


313 


27.1 


29.5 


-2.4 


-8.0 


Riverside Human Capital Development 


406 


28.3 


29.7 


-1.4 


-4.8 




Used a nonreeular or sporadic child care 






arraneement in the past week (Vo) 




Atlanta Labor Force Attachment 


580 


4.4 


3.6 


0.8 


23.3 


Atlanta Human Capital Development 


650 


3.3 


3.6 


-0.3 


-7.5 


Grand Rapids Labor Force Attachment 


394 


4.1 


5.4 


-1.3 


-23.9 


Grand Rapids Human Capital Development 


380 


7.4 


4.3 


3.0 


69.6 


Riverside Labor Force Attachment 


493 


5.6 


9.4 


-3.8 


-40.2 


Lacked high school diploma or basic skills 


313 


5.9 


10.4 


-4.5 


-43.4 


Riverside Human Capital Development 


406 


8.9 


10.2 


-1.3 


-13.1 




Used multiple reeular child care arrangements 






in the past week (Vo) 




Atlanta Labor Force Attachment 


580 


5.2 


6.2 


-1.0 


-16.9 


Atlanta Human Capital Development 


650 


4.1 


6.2 


-2.1 


-33.3 


Grand Rapids Labor Force Attachment 


394 


5.3 


7.2 


-1.8 


-25.5 


Grand Rapids Human Capital Development 


380 


3.8 


6.8 


-3.1 


-44.9 


Riverside Labor Force Attachment 


493 


8.0 


9.6 


-1.6 


-17.0 


Lacked high school diploma or basic skills 


313 


6.6 


7.0 


-0.4 


-5.5 


Riverside Human Capital Development 


406 


7.9 


7.1 


0.8 


11.2 



(continued) 
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Table 10.2 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control Difference 
Group (Impact) 


Percentage 
Change (%) 




Number of hours child spent in all regular 






care arrangements in the past week 




Atlanta Labor Force Attachment 


580 


9.1 


10.3 


-1.2 


-11.6 


Atlanta Human Capital Development 


650 


8.7 


10.1 


-1.3 


-13.3 


Grand Rapids Labor Force Attachment 


392 


11.3 


10.3 


1.1 


10.6 


Grand Rapids Human Capital Development 


378 


7.5 


10.0 


-2.6 


-25.4 


Riverside Labor Force Attachment 


492 


8.4 


6.6 


1.7 


26.4 


Lacked high school diploma or basic skills 


312 


6.7 


5.6 


1.1 


19.0 


Riverside Human Capital Development 


406 


5.5 


6.0 


-0.5 


-8.4 




Amons those usins reeular care arransements 






in the past week, number of hours used 




Atlanta Labor Force Attachment 


232 


22.1 


25.6 






Atlanta Human Capital Development 


249 


23.2 


25.2 






Grand Rapids Labor Force Attachment 


166 


26.9 


24,0 






Grand Rapids Human Capital Development 


147 


20,1 


24,5 






Riverside Labor Force Attachment 


166 


25.3 


19,5 






Lacked high school diploma or basic skills 


89 


26.4 


18,2 






Riverside Human Capital Development 


118 


19,8 


20.0 










Anv self-care in the past week 




Atlanta Labor Force Attachment 


578 


10.1 


9.8 


0.3 


3.1 


Atlanta Human Capital Development 


649 


9.5 


9.7 


-0.3 


-2.6 


Grand Rapids Labor Force Attachment 


392 


10.8 


11.6 


-0.8 


-6.9 


Grand Rapids Human Capital Development 


375 


12.8 


11.5 


1.3 


11.6 


Riverside Labor Force Attachment 


All 


9.8 


8.5 


1.3 


15.9 


Lacked high school diploma or basic skills 


304 


8.2 


4.9 


3.2 


65.1 


Riverside Human Capital Development 


396 


5.7 


5.2 


0.5 


10.2 
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Table 10.2 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference Percentage 

(Impact) Change (%) 




Used a orovider aeed 17 or under in the main 








child care arrangement in the oast week (Vo) 




Atlanta Labor Force Attachment 


580 


3.9 


3.4 


0.6 


16.3 


Atlanta Human Capital Development 


650 


3.9 


3.7 


0.2 


5.7 


Grand Rapids Labor Force Attachment 


394 


4.1 


5.4 


-1.3 


-24.2 


Grand Rapids Human Capital Development 


380 


2.9 


. 5.3 


-2.4 


-45.9 


Riverside Labor Force Attachment 


492 


2.9 


4.8 


. -1.9 


-39.0 


Lacked high school diploma or basic skills 


313, 


2.0 


4.7 


-2.7 


-57.0 


Riverside Human Capital Development 


406 


4.1 


4.8 


. -0.7 


-14.9 




Main arrangement used in the oast week was formal (Vo) 


Atlanta Labor Force Attachment 


580 


18.6 


19.4 


-0.7 


-3.8 


Atlanta Human Capital Development 


650 


18.2 


18.9 


-0.6 


-3.4 


Grand Rapids Labor Force Attachment 


394 


8.2 


10.4 


-2.3 


-21.8 


Grand Rapids Human Capital Development 


380 


6.1 


10.2 


-4.1 


-40.6 


Riverside Labor Force Attachment 


492 


2.6 


8.0 


-5.4 ♦♦ 


-67.2 


Lacked high school diploma or basic skills 


312 


1.4 


3.9 


-2.5 


-64.4 


Riverside Human Capital Development 


405 


2.3 


4.0 


-1.7 


-42.2 




Main arrangement used in the oast week was informal (Vo) 


Atlanta Labor Force Attachment 


580 


26.5 


23.8 


2.7 


11.3 


Atlanta Human Capital Development 


650 


22.2 


23.9 


-1.8 


-7.3 


Grand Rapids Labor Force Attachment 


394 


35.2 


36.0 


-0.9 


-2.4 


Grand Rapids Human Capital Development 


380 


31.0 


34.4 


-3.4 


-9.8 


Riverside Labor Force Attachment 


492 


33.1 


28.9 


4.2 


14.5 


Lacked high school diploma or basic skills 


312 


27.0 


27.8 


-0.8 


-3.0 


Riverside Human Capital Development 


405 


26.9 


28.0 


-1.1 


-3.9 
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Table 10.2 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Percentage 
Change (%) 






Reported child care as a barrier 








to schooU iob training, or work (%) 




Atlanta Labor Force Attachment 


575 


15.4 


16.3 


-0.9 


-5.3 


Atlanta Human Capital Development 


645 


16.2 


16.5 


-0.3 


-1.6 


Grand Rapids Labor Force Attachment 


392 


30.0 


30.8 


-0.8 


-2.6 


Grand Rapids Human Capital Development 


377 


25.2 ’ 


29.2 


-4.1 


-13.9 


Riverside Labor Force Attachment 


489 


27.4 


21.2 


6.2 


29.3 


Lacked high school diploma or basic skills 


312 


21.6 


18.1 


3.5 ' 


19.6 


Riverside Human Capital Development 


402 


20.8 


18.7 


2.0 


10.8 



SOURCE: Child Trends calculations from the Five-Year Child Outcomes Study survey (mother reports). 



NOTES: See Appendix A.2. 
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tal hours in child care across all regular airangements did not differ in any of the LFA or HCD 
programs, nor did reliance on multiple arrangements. 

There was very little evidence of a shift in type of arrangements relied upon. Notably, 
there were no program impacts on any use of self-care or on reliance on a child care provider 
aged 17 or under in the primary arrangement (but see the findings below on time spent with only 
a young peer or sibling when mothers reported time use on a recent weekday). Thus, there is no 
indication of an increase in reliance on these potentially problematic arrangements by LFA or 
HCD program group members when considering child care arrangements in the week prior to the 
interview. 

With a single exception (the Riverside LFA program diminished reliance on formal child 
care), there were no impacts on use of formal and informal child care as the main arrangement. 

2. Supervision and Activities for the Focal Child 

Findings will now be presented for the measures of with whom the focal child spent his 
or her time on a recent weekday afternoon and evening. Because mothers were also asked to re- 
port whether the child was in a child care setting for this time period (child care in a center or 
program, child care by a babysitter, or care in either context on a drop-in basis), it is possible to 
directly juxtapose impacts on child care and impacts on mothers’ reports of whom the child was 
with. For example, it will be possible to identify whether there were impacts on supervision 
(whom the child was with) even in the absence of impacts on measures of child care. This set of 
questions again addresses whether children spent time in self-care, here also asking about time 

spent with only a young peer or sibling (under age 13). Finally, this section reports on participa- 

26 

tion in activities or lessons (such as a sport, club, or music lesson) during after-school hours. 
(See Table 10.3.) 

There were virtually no impacts on any of these variables in the Grand Rapids and At- 
lanta programs. However, there was a cluster of findings for both programs in Riverside. They 
indicate a decrease in time with mother and an increase in time with an adult other than the 
mother in both programs. In the aggregate, none of the programs increased the proportion of 
children who spent time in after-school activities or who spent time unsupervised by an adult. 
For the subgroup of families in which the mother lacked basic education at baseline in Riverside, 
the LFA program increased the proportion of children who spent time without an adult present 
(though a sizable difference, this was just above the cutoff for significance). These findings are 
presented in greater detail below. 

3. Impacts on the Riverside Programs 

The LFA program: all families. For all families assigned to the LFA program in River- 
side (irrespective of mothers’ educational level at baseline), focal children in the program group 
spent significantly less time with their mothers and more time with an adult other than the 



^^Results are summarized in terms of the number of half-hour periods (out of a possible 18) between 3 P.M. and 
midnight on the chosen weekday that a child was with someone or engaging in a particular care situation or activity. 
Where distributions were limited, results are summarized instead in terms of the proportion of children for whom this 
companion or activity was reported at all. 
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National Evaluation of Welfare-to-Work Strategies 
Table 10.3 

Impacts on Activities Engaged in by the Focal Child on a Recent Weekday 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Percentage 
Change (%) 




Number of half-hour time periods spent with the mother 






tO-18 periods from 3 pm to midnight) 




Atlanta Labor Force Attachment 


578 


12.9 


12.7 


0.2 


1.6 


Atlanta Human Capital Development 


649 


13.2 


12.7 


0.5 


3.6 


Grand Rapids Labor Force Attachment 


392 


12.5 


12.2 


0.3 


2.5 


Grand Rapids Human Capital Development 


376 


13.0 


12.2 


0.8 


6.8 


Riverside Labor Force Attachment 


477 


13.4 


14.3 


-0.9 ♦ 


-6.3 


Lacked high school diploma or basic skills 


304 


13.9 


14.8 


-0.9 ♦ 


-6.2 


Riverside Human Capital Development 


396 


14.1 


14.9 


-0.8 ♦ 


-5.5 




Number of half-hour time periods spent with an adult 




other than the mother tO-lS periods from 3 pm to midnight) 


Atlanta Labor Force Attachment 


578 


3.6 


3.7 


-0.1 


-3.7 


Atlanta Human Capital Development 


649 


3.2 


3.7 


-0.4 


-12.0 


Grand Rapids Labor Force Attachment 


392 


4.4 


4.5 


-0.1 


-2.0 


Grand Rapids Human Capital Development 


376 


4.2 


4.5 


-0.3 


-6.5 


Riverside Labor Force Attachment 


477 


3.5 


2.5 


0.9 ♦♦ 


37.4 


Lacked high school diploma or basic skills 


304 


2.8 


2.1 


0.6 


29.8 


Riverside Human Capital Development 


396 


2.9 


2.1 


0.9 ♦ 


42.0 






Anv time spent without an adult present 








from 3 p.m. to midnight t%) 




Atlanta Labor Force Attachment 


578 


11.5 


12.7 


-1.1 


-9.0 


Atlanta Human Capital Development 


649 


12.1 


13.1 


-1.0 


-7.5 


Grand Rapids Labor Force Attachment 


392 


5.4 


7.2 


-1.8 


-24.5 


Grand Rapids Human Capital Development 


376 


3.5 


7.3 


-3.8 


-51.7 


Riverside Labor Force Attachment 


477 


11.1 


10.2 


0.9 


8.9 


Lacked high school diploma or basic skills 


304 


15.2 


9.8 


5.5 


55.9 


Riverside Human Capital Development 


396 


10.2 


9.6 


0.6 


6.3 



(continued) 
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Table 10.3 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference Percentage 

(Impact) Change (%) 




Number of half-hour time periods spent with 








only a young peer or sibling 






(under aee 13, 0-18 periods from 3 pm to midniehtl 




Atlanta Labor Force Attachment 


578 


0.9 


0.9 


0.0 


2.9 


Atlanta Human Capital Development 


649 


0.9 


0.9 


0.1 


7.4 


Grand Rapids Labor Force Attachment 


392 


0.8 


0.9 


-0.2 


-17.2 


Grand Rapids Human Capital Development 


376 


0.5 


0.9 


1 

o 

# 


-39.4 


Riverside Labor Force Attachment 


477 


0.7 


0.6 


0.1 


24.0 


Lacked high school diploma or basic skills 


304 


0.7 


0.5 


0.2 


30.5 


Riverside Human Capital Development 


396 


0.6 


0.6 


0.1 


11.7 






Any time periods in child care 






(sitter, center or proeram. 


or drop-in child care) 








from 3 p.m. to midnight (%) 




Atlanta Labor Force Attachment 


580 


15.1 


15.5 


-0.5 


-3.0 


Atlanta Human Capital Development 


650 


10.6 


15.3 


-4.7 ♦ 


-30.5 


Grand Rapids Labor Force Attachment 


394 


14.2 


18.7 


-4.5 


-24.1 


Grand Rapids Human Capital Development 


380 


14.2 


18.2 


-4.0 


-21.8 


Riverside Labor Force Attachment 


493 


1.4 


4.4 


-2.9 ♦ 


-67.0 


Lacked high school diploma or basic skills 


313 


1.6 


1.5 


0.1 


10.0 


Riverside Human Capital Development 


406 


5.8 


1.7 


4.0 ♦♦ 


232.0 




Anv time periods in an activity or lesson 








from 3 p.m. to midnight (%) 




Atlanta Labor Force Attachment 


580 


8.2 


8.4 


-0.2 


-2.3 


Atlanta Human Capital Development 


650 


7.5 


8.5 


-1.0 


-11.2 


Grand Rapids Labor Force Attachment 


394 


3.9 


6.1 


-2.3 


-37.0 


Grand Rapids Human Capital Development 


380 


5.8 


5.5 


0.3 


5.3 


Riverside Labor Force Attachment 


493 


2.8 


6.0 


-3.2 


-53.2 


Lacked high school diploma or basic skills 


313 


1.8 


5.5 


-3.7 


-67.8 


Riverside Human Capital Development 


406 


2.2 


4.9 


-2.6 


-54.1 



SOURCE: Child Trends calculations from the Five-Year Child Outcomes Study survey (mother reports). 



NOTES: See Appendix A.2. 



mother. These impacts did not reflect large differences in amounts of time spent with the mother 
and another adult, however, averaging about a single half-hour less with the mother and about a 
single half-hour more with another adult. 

This increase in time spent with another adult was reported in the absence of an impact on 
mothers’ report of child care use during this period. The impact noted above on type of care for 
the Riverside LFA program can now be placed in a larger context. In this program, there was ap- 
parently a shift in type of care away from reliance on formal child care (detected in the measure 
of child care use in the past week) and toward reliance on another adult to supervise the child in a 
situation mothers did not describe as child care. 

The LFA program: families in which the mother lacked a high school diploma or 
GED at baseline. For families in the LFA program in which the mother lacked a high school di- 
ploma or GED at baseline, the pattern was similar to that for all LFA families in that program 
group children spent less time with their mothers (with the difference in time periods again just 
under a single half-hour) and were less likely to be in a formal child care setting on a regular ba- 
sis (though the difference in formal child care use was just above the cutoff for statistical signifi- 
cance in this sample). However, unlike those in the full LFA sample, children of mothers in the 
LFA program who lacked this credential at baseline did not spend more time with another adult. 
Rather, they were more likely than the control group and the HCD program group to have spent 
time without an adult present, though this sizable difference, comparable to impacts on other 
measures, was just above the cutoff for statistical significance. This is the one indication of a 
finding pointing to a program increasing self-care, and it is of concern that it occurs in a higher- 
risk subgroup. 

The HCD program. Children of HCD program group members spent less time with the 
mother and more time with another adult than their control group counterparts. These differences 
also involved only about a single half-hour. However, children of HCD program group members 
were more likely than their control group counterparts to be in what the mother described as child 
care (including center or program care, a babysitter, or drop-in care in either setting). Indeed, 
children of mothers in the HCD program were also significantly more likely than children of 
mothers with limited education in the LFA program to be in child care on a recent weekday. 

The clustering of impacts on measures of supervision and child care in the Riverside site 
is in accord with impacts on employment in the Child Outcomes Study sample. The measure of 
mothers’ employment that is most closely aligned in terms of time frame is a survey measure of 
full-time employment at the time of the survey. Impacts on this measure were statistically signifi- 
cant only in the Riverside site and were moderate to large in size.^^ For families in the Riverside 
LFA program (irrespective of mothers’ educational level at baseline), there was a difference of 
1 1 percentage points in reported current full-time employment (40.6 percent of program group 
mothers and 29.6 percent of control group mothers reported current full-time employment) and 
the impact for those in the LFA program who lacked a high school diploma or GED was 15.5 
percentage points (39.2 of program group members compared to 23.7 of control group members 
reporting employment). The impact for the Riverside HCD program was 9 percentage points 



^’See Appendix I. 




- 217 - 



162 



(32.7 percent of program group mothers and 23.7 percent of control group mothers reported cur- 
rent full-time employment). Thus, the child care and activities findings correspond closely to im- 
pacts on concurrent employment, specifically full-time employment as reported by the mother, in 
the Riverside site. 

Further impacts on adult outcomes in the Child Outcomes Study sample may also be 
relevant. At the time of the five-year follow-up. Riverside program group members were more 
likely than control group members to be cohabiting (significant for the full LFA group; not 
shown), to have a new baby (significant for the full LFA group and also for those in the LFA 
group lacking a high school diploma or GED at baseline), and to be living with a partner or 
spouse as well as children (not shown). In the Riverside LFA group, program group children 
spent less time with the mother and more time with another adult (though not in the context of 
what the mother describes as child care). One interpretation of these findings is that program 
group children more often than control group children were cared for in their mother’s absence 
by the mother’s spouse or cohabiting partner. 

V. Conclusions 

Impact findings for child care differed according to the time period considered. Across 
the five-year follow-up, most programs showed impacts on mother’s reliance on child care 
while leaving welfare because of earnings, and several programs showed impacts on use of 
transitional child care benefits. However, for recent child care use, there were few impacts. 
Fortunately, this pattern of very few impacts on concurrent measures of child care and supervi- 
sion includes the measures of self-care and being only with a young peer or sibling. Unfortu- 
nately, there is no indication here of the possibly salutary influence of participation in after- 
school activities and lessons.^^ 

For recent child care, it is primarily when going beyond traditional measures of child care 
that a pattern emerged. When asking about whom the child was with on a recent weekday after- 
noon, findings indicated a slight shift away from time with the mother and toward time with an- 
other adult in both Riverside programs. In the LFA program, this was accompanied by dimin- 
ished participation in formal child care as the primary arrangement. 

In sum, in most of the programs the picture that emerges is of intermittent or weak im- 
pacts, captured best by measures across time and considering more than one child in a family. 
Only in Portland were impacts apparent for a recent spell of employment, and only in Riverside 
was there evidence for the focal child of impacts occurring on type of care and nature of supervi- 
sion at the time of the final follow-up.^° 



^*See Appendix I. 

noted above, while findings of one recent evaluation suggest that participation in after-school child care can 
be beneficial for children from low-income families, implications for children are likely to vary with the quality of 
care. 

^®The information collected in Atlanta, Grand Rapids, and Riverside on child care and child activities on a re- 
cent weekday was not collected in Portland. 
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What do these findings indicate regarding child care as a support for employment in these 
welfare-to-work programs? The fact that child care impacts mirrored the broad pattern of em- 
ployment findings, of cumulative effects rather than widespread effects at the time of the final 
follow-up, suggests that child care use was broadly coordinated with employment, and often 
functioning to enable employment. Families with an employed mother using child care tended to 
have out-of-pocket expenses for child care. Yet there were few impacts on the use of transitional 
child care benefits, and overall use of these benefits was fairly low and quite variable across sites. 
There are several possible interpretations of this finding, including the possibilities that restric- 
tions on the type of care that could be subsidized with these benefits, limited outreach and infor- 
mation about them provided by caseworkers, or the short period (one year) in which benefits 
could be used after moving from welfare limited families’ abilities to make use of these benefits. 
Also, it is important to note that across sites there were differences in the types of child care that 
were encouraged and facilitated. For example, staff members in Atlanta encouraged use of for- 
mal child care by reimbursing welfare recipients only for use of licensed or certified care. In con- 
trast, staff in Riverside encouraged informal child care arrangements by emphasizing that welfare 
recipients should choose child care that they would be able to afford once they were no longer 
eligible for benefits. 

What of child care as a context for children’s development? The findings here cannot re- 
flect on the critical issue of child care quality, but do provide a picture of the extent of children’s 
exposure to child care, type of care, use of self-care, and exposure to after-school activities and 
lessons. Findings generally do not point to elevations in potentially harmful forms of care such as 
self-care or care by a young caregiver, though findings may suggest the need to continue to moni- 
tor this issue. At the same time, they do not point to systematic increases in the possibly benefi- 
cial environments of after-school activities and lessons or in use of formal child care arrange- 
ments. Rather, children appear to have received an elevated “dose” of child care over time in 
most programs, but as indicated in the measures of recent child care, this dose does not appear to 
have been large or sustained in most programs. However, in Riverside (as in Portland) a few 
findings point to more sustained child care impacts. Some findings in the Riverside LFA program 
also point to a shift away from formal child care arrangements and toward informal care that the 
mother may not describe as a child care arrangement. Chapter 12 returns to the possibility that 
this pattern of child care impacts may help to explain impacts on child outcomes, with a particu- 
lar focus on findings in the Riverside site, where the impacts extended to current supervision and 
child care, as well as type of care. 
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Chapter 11 

Impacts on the Well-Being of All Children 



The effects of welfare-to-work programs on child well-being are addressed in two chap- 
ters in this report. This chapter captures the breadth of the effects of seven welfare-to-work pro- 
grams on the well-being of children of all respondents in the Five-Year Client Survey. Program 
effects are evaluated for children in four age groups, from toddlers to adolescents at the time of 
study entry, and on outcomes in two domains of child development — academic functioning and 
health and safety — as well as on other selected outcomes. Chapter 12 analyzes, in greater depth, 
effects on child well-being at the five-year follow-up for a subset of “focal” children, aged 3 to 5 
at random assignment, in six welfare-to-work programs in Atlanta, Grand Rapids, and River- 
side.' Details about the samples examined in these two chapters and how they are derived from 
the full impact sample in this report are shown in Figure 11.1. 

As discussed in Chapter 1, a central goal of the federal JOBS program, under the 1988 
Family Support Act (FSA), was to move single mothers from public assistance to paid employ- 
ment. This goal was largely implemented by imposing strict participation and work requirements. 
For 20 years prior to 1988, women receiving welfare who had children under age 6 generally 
were not subject to these mandates. With the passage of the FSA, women with children as young 
as age 3 (or as young as age 1, at state option) were newly designated as mandatory participants. 
Thus, in the early 1990s there was much interest in how welfare-to-work programs might affect 
children, especially very young children. The well-being of children remained of central concern 
and produced considerable debate at the passage of the more recent 1996 welfare reform law 
(Personal Responsibility and Work Opportunity Reconciliation Act, or PRWORA) that similarly 
imposed requirements to move welfare recipients into employment and imposed participation 
mandates on women with children as young as age 1 (or younger, at state option). A two-year 
follow-up of the effects of the NEWWS programs on child well-being, which included a special 
study of preschool-age children, was one of the first to inform this debate. The analysis in these 
two chapters represents the first evaluation of long-iQxm effects of mandatory welfare-to-work 
programs on the well-being of children. 

Earlier chapters in this report primarily focused on the effects of welfare-to-work pro- 
grams on the outcomes targeted by these programs, such as education, employment, and welfare 
receipt. As these chapters revealed, during the five-year follow-up period the NEWWS programs, 
regardless of whether they were employment- or education-focused, generally increased partici- 
pation in employment-related activities, employment and earnings and reduced public assistance 
payments. The consistency of these findings across programs and sites is not surprising given the 
stated goals of the programs. Appendix I summarizes these effects for the Five-Year Client Sur- 
vey sample and shows that, with some exceptions for the Riverside LFA and HCD programs, 
and, most important, the Portland program, the pattern of impacts for the respondents in the Five- 



'This chapter presents program impacts on selected outcomes for all client survey respondents’ children aged 3 
to 5. See Chapter 12 for more detail about how the Child Outcomes Study (COS) sample differs from this sample. 
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Figure 11.1 

Samples and Subsamples Used in Chapters 11 and 12 
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Year Client Survey sample were quite similar to the impacts for the full impact sample presented 
and discussed in Chapters 4-6. 

Unlike early childhood intervention programs, the NEWWS programs were not structured 
to directly affect the well-being of children. It is still possible, however, that welfare-to-work 
programs could produce either favorable or unfavorable effects on child outcomes. Current theo- 
ries hypothesize that, by affecting the behavior of mothers’ programs could also indirectly affect 
children’s well-being through, for example, changes in mothers’ psychological well-being or par- 
enting styles, in child care, and/or in family life and material resources. Some of these changes 
may bode well for certain child outcomes but prove problematic for others, and thus the effects 
of a given type of welfare-to-work program on child outcomes may not be uniformly favorable or 
unfavorable across developmental domains.^ Also, it is possible that program impacts on child 
outcomes may vary, depending on whether the effects are enduring, the extent of the exposure to 
the program, or on the combined effects of all program impacts. 

The ways that welfare programs might affect child well-being can be depicted in a simple 
conceptual framework.^ A program’s features, such as its message, sanctioning rates, and moni- 
toring, can affect the targeted or direct outcomes of the program, such as employment, public as- 
sistance receipt, income, and education, and nontargeted outcomes, such as child care and parent- 
ing behavior, and ultimately lead to effects on child outcomes. Prior research, though largely 
based on nonexperimental studies, provides a basis for predicting and understanding how effects 
on key outcomes such as education, employment, earnings, and income may affect children. For 
example, mothers’ increased educational attaimnent and employment (depending on the extent 
and quality of the job), and/or family income may prove beneficial to children in low-income 
families.'* Mothers’ psychological well-being and parenting — shown to be related to children’s 
developmental outcomes in the nonexperimental literature^ — may also be affected by these pro- 
grams, though the extent and direction of likely change may vary.^ Child care is another way in 



^For example, while research generally shows better cognitive and behavioral outcomes for children in formal 
child care settings (NICHD Early Child Care Research Network, 2000; Zaslow et al., 1998), studies have also linked 
attendance in larger child care settings with greater ear infections, with detrimental effects on hearing loss and lan- 
guage development more likely to occur in low-quality settings (Vemon-Feagans, Emanuel, and Blood, 1997). Thus, 
mothers’ involvement in welfare-to-work programs that leads to increased use of large, formal settings for their 
young children may benefit their cognitive and behavioral development but adversely affect their health. In addition, 
any one program may have different effects on child outcomes, and these effects may reinforce or cancel each other 
out. For example, any beneficial effect of mothers’ increased employment may be attenuated by any declines in in- 
come. 

^See McGroder et al., 2000; Hamilton, 2000; Zaslow et al., 1998, 1995. 

“For various reviews about the effects of income on children’s outcomes see Duncan and Brooks-Gunn, 1997; 
Harvey, 1999; Hoffinan and Youngblade, 1999; Mayer, 1997; Moore and Driscoll, 1997; Vandell and Ramanani 
1992; and Zaslow et al., 1999a. Some argue that the effects of family income may also depend on the source of in- 
come, though many studies find no relation between welfare receipt and children’s development controlling for 
demographic and family characteristics (for example, Haveman and Wolfe, ^1995; Levine and Zimmerman, 2000; 
Yoshikawa, 1999). 

^McGroder, 2000. 

®See, for example, -the “stress hypothesis” proposed by Brooks-Gunn and Berlin, 1993. See also Zaslow et al 
1995. 
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which mothers’ employment may affect children. Unstable or low-quality child care may produce 
detrimental effects on children’s development.^ 

This chapter reviews general hypotheses and impact findings on child outcomes, organ- 
ized by child age. It discusses whether or not mothers with children in various age groups may 
have behaved and responded differently to these welfare-to-work programs. ’ Also, welfare-to- 
work programs might have affected children differently at different points in their development. 
For example, toddlers may be the most vulnerable to possible negative effects of mothers’ em- 
ployment, particularly if they are placed in poor-quality child care. Adolescents, in contrast, may 
have the most to gain if they are placed in enriching after-school programs. Older children may 
be left unsupervised or may take on more responsibilities at home as mothers join the workforce, 
which could lead to unfavorable effects on their development, particularly their social behavior. 
In reviewing the discussion of impacts throughout this chapter, it is important to remember that, 
as discussed in Chapter 1, some control group members in Atlanta and Grand Rapids became eli- 
gible for program services after the third year of the follow-up period. However, it appears that 
program impacts on earnings and welfare receipt were only slightly affected by the end of the 
control group embargo in these sites. The chapter ends with a discussion, again by child age, of 
what may have led to program effects on children. 

Table 11.1 summarizes the impacts on the limited set of outcomes examined in this chap- 
ter, by child age. 

I. Key Findings . 

• There is little evidence that these welfare-to-work programs had long- 
term effects on children in general. Effects that did emerge were clustered 
by age of the child, with most effects occurring among children who were 
adolescents at the start of the study (aged 15 to 23 at the five-year follow- 
up). There is less convincing evidence that the effects of these welfare-to- 
work programs on children are clustered or vary by domain of child develop- 
ment, program approach, or site. 



’Lamb, 1998; Phillips et al., 1994; Zaslow, 1991. 

^Survey respondents may have children who fall into one or more age groups; for example, a respondent may 
have a preschool-age child and a young school-age child at study entry. For this reason, siuwey impacts on economic 
outcomes for the mothers of children in various age groups do not necessarily reflect effects on mutually exclusive 
groups. 

^Families with children in different age groups may also differ in other ways. For example, respondents with 
adolescent children are more likely to be older, more likely to have been ever married, less likely to have a high 
school diploma or a GED, and more likely to be a longer-term recipient of welfare than respondents with younger 
children. Program impacts may instead reflect other characteristics of these respondents. Conditional subgroup im- 
pact analysis, which would test whether or not age of children or other characteristics explain program impacts, is 
beyond the scope of work for this chapter. 

'°In general, there were a greater number of statistically significant findings than expected by chance alone over- 
all and by each of the child age groups. See Chapter 2 for a more detailed description of the standard used to deter- 
mine the likelihood of chance findings. 
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National Evaluation of Welfare-to-Work Strategies 
Table 11.1 



Summary of Impacts on Child Outcomes, by ChUd Age at Random Assignment 







Academic Functioning 




Health and Safety 


Other 
















Had a 
















Had a 


Condition 


Did Not 














Condition 


That 


Live With 














That 


Impeded on 


Mother 










Attended a 




Required 


Mother's 


Because 










Special 


Dropped 


Frequent Required Ability to Go 


She Could 


Had a 


Repeated 


Suspended Class for a 


Out of 


Medical Emergency 


to Work or 


Not Care Baby as a 


a Grade 


or Expelled 


Condition 


School 


Attention Room Visit 


School 


for Child 


Teen 


Toddlers 


















(aged 6 and 7 at follow-up) 
















Grand Rapids LFA 




F 






F 








Grand Rapids HCD 




F 




- 








- 


Portland 








- 








- 


Preschool-age children 
(aged 8 to 10 at follow-up) 


















Atlanta LFA 








_ 


U 


U 






Atlanta HCD 








- 


U 






- 


Grand Rapids LFA 








_ 










Grand Rapids HCD 


U 






- 








- 


Riverside LFA 


F 






_ 


f 


F 


S 




In need® 


F 






- 




F 






Riverside HCD 








- 








- 


Portland 








- 








- 


Young school-age children 


















(aged 11 to 14 at follow-up) 
















Atlanta LFA 


















Atlanta HCD 
















- 


Grand Rapids LFA 


u 


U 














Grand Rapids HCD 










F 


F 




- 


Riverside LFA 




F 










S 




In need® 


u 


F 










S 




Riverside HCD 




F 






F 




S 


- 


Portland 
















- 


Adolescents 


















(aged 15 to 23 at follow-up) 
















Atlanta LFA 
Atlanta HCD 




F 












f 


Grand Rapids LFA 


U 




u 




u 








Grand Rapids HCD 


U 




u 












Riverside LFA 


U 




u 




F 




S 




In need® 


u 




U 




F 




s 


U 


Riverside HCD 


U 




U 


U 




U 




u 


Portland 
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(continued) 



Table 11.1 (continued) 



NOTES: "F" indicates a statistically significant favorable impact. "U" indicates a statistically significant unfavorable impact, 
"f indicates a favorable impact above the cutoff for statistical significance but part of the overall pattern, "u" indicates an 
unfavorable impact above the cutoff for statistical significance but part of the overall pattern. "S" indicates a statistically 
significant impact that could not be categorized as favorable or unfavorable, "s" indicates an impact above the cutoff for 
statistical significance but part of the overall pattern. See Chapter 2 for the definition of a pattern. - indicates that a measure 
was not appropriate for a particular child age group. Blank spaces indicate that there were no impacts. 

“Sample members lacked a high school diploma or basic skills at random assignment. 



• There were few differences in the effects on child outcomes by program 
approach. 

• Few program effects were found for children who were toddlers at the 
start of the study (aged 6 and 7 at the five-year foIIow-up). It is notewor- 
thy, however, that on the few outcomes measured in the two sites with 
data for children in this age group more unfavorable effects were not 
found. In fact, both Grand Rapids programs produced a consistent pattern of 
favorable effects (though not always statistically significant and thus not al- 
ways shown in Table 11.1). Perhaps one explanation is that the Grand Rapids 
programs were more likely to increase part-time employment for the mothers 
of these toddlers. A similar pattern of effects on part-time employment was 
not found in Portland or for mothers with older children. 

• Few program effects were found for children who were preschool-age or 
young school-age at the start of the study (aged 8 to 10 and 11 to 14, re- 
spectively, at the five-year follow-up). Any effects that did occur for these 
children were both favorable and unfavorable. 

• The effects of the welfare- to- work programs on adolescents who were 
aged 15 to 23 at the five-year follow-up were generally unfavorable for 
children with parents in the Grand Rapids and Riverside programs, par- 
ticularly in the Riverside HCD program. These effects were concentrated in 
the academic functioning domain and included increased grade repetition, in- 
creased attendance in a special class, and, in one program, increased likeli- 
hood of dropping out of school. Mothers of adolescents in these two sites ex- 
perienced the largest increases in employment during the first year of follow- 
up, decreased average income from earnings and welfare benefits, and, in two 
programs, were more likely to be married at the five-year follow-up point. 
Adolescents academic functioning may have been especially vulnerable to 
any one or all of these changes. 

II. Measurement Issues 



The measures of child well-being analyzed in this section were constructed from mothers’ 
responses to the Five-Year Client Survey. Respondents were asked about their children’s academic 
functioning, health and safety, and other outcomes. (See Table 11.1.) Respondents answered for 
their own children — whether biological children, legally adopted children, or stepchildren. Al- 
though they were asked about all children regardless of age, this analysis is limited to children who 
were aged 18 or under at random assignment. This section provides a discussion about the construc- 
tion of these measures. (Details about these measures can also be found in Appendix J.) 

Respondents were asked a variety of questions about their children, concerning events or 
situations since a focal date. For most respondents the focal date was the date of the two-year in- 
terview. For those who did not answer the two-year survey, the focal date was the date of random 
assignment. Thus, for most respondents, child outcomes are measured for years 3 to 5 of follow- 
up (that is, since the two-year interview point). Some outcomes, such as dropping out of school. 



were not restricted to the period of time since the focal date. Instead these outcomes were meas- 
ured “ever” in a child’s life. 

The first group of measures captures children’s academic functioning including whether 
or not they had been suspended or expelled from school, repeated a grade, dropped out of school, 
or attended a special class or special school for a physical, emotional, or mental condition. 

The second group of measures captures children’s health and safety, including whether 
they had a physical, emotional, or mental condition that required frequent medical attention, fre- 
quent use of medication, or the use of any special equipment; whether they ever had a physical, 
emotional, or mental condition that demanded a lot of attention and made it difficult for the re- 
spondent to attend work or school; and whether they had an accident, injury, or poisoning that 
required a visit to a hospital emergency room or clinic. This last outcome is only a rough proxy 
for child safety. On the one hand, it may measure neglect if children are experiencing more acci- 
dents or injuries as mothers increase their work effort and children are left unsupervised, with 
abusive partners or other adults, or, possibly, placed in unsafe child care arrangements. On the 
other hand, this outcome may simply reflect a mother’s ability to purchase medical care. Emer- 
gency room use may be used as a replacement for visits to a doctor or a clinic. 

The final group of measures concerns living arrangements (whether a child did not live 
with their mother because she could not care for them) and teenage parents (whether a child gave 
birth as a teenager). In this analysis children were considered to have been teenage parents if they 
had a child at age 18 or under during the five-year follow-up period. Whether children did not 
live with their mother because she could not care for them may have numerous interpretations: It 
may capture the consequence or result of a government intervention in the family, for example, 
being forced to place children in foster care. However, it is also possible that a mother voluntarily 
elected to place her children in another living arrangement, which may provide a better environ- 
ment. This outcome may be also interpreted more generally as a measure of household composi- 
tion or living arrangements rather than a direct measure of child well-being. 

Outcomes in this chapter are presented for any child in the family and for each child in 
the family. Measures presented for any child indicate the percentage of families in which at least 
one child experienced a certain outcome (for example, “any child ever repeated a grade” indi- 
cates that at least one child in the family repeated a grade) and provide a general snapshot of 
child outcomes at the family level. These measures are similar to those analyzed at the two-year 
follow-up point. Measures presented for each child in the family indicate the percentage of chil- 
dren who have experienced a certain outcome. Thus, unlike most of the information collected in 
the survey, these outcomes are specific to a child within a family. Each child in a family who was 
18 or under at random assignment is represented in the impact analysis." The 5,342 families in 
this analysis had a total of 13,726 children. 



' 'All standard errors in the impact analysis were adjusted so that the impact estimates account for the presence 
of two or more children or siblings within a family. 
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Four age groups of children are examined: (1) children who were toddlers at random as- 
signment (aged 6 and 7 at the five-year follow-up; 7 percent were age 8);'^ (2) preschool-age 
children (8 to 10 at the five-year follow-up; 4 percent were age 11); (3) young school-age chil- 
dren (aged 1 1 to 14 at the five-year follow-up; 3 percent were age 15); and (4) adolescents (aged 
15 to 23 at the five-year follow-up; 78 percent were age 15 to 20; 22 percent were 21 to 23, and 1 
percent were 24). 

The ages of respondent’s children varied across sites. As discussed in Chapter 1, mothers 
with children aged 3 or over were required to participate in most sites. However, in Grand Rapids 
and Portland, the participation mandate was extended to single mothers with a child as young as 
age 1. Although this analysis treats children mutually exclusively, the mothers of these children 
within and across each age group are not mutually exclusive: 41.5 percent of respondents have 
more than one child within one age group and 49.6 percent have at least one child in more than 
one age group. The sample size for each of these age categories and the general stmcture of how 
the samples were derived for these two chapters are presented in Figure 11.1. 

Although the outcomes covered in this chapter provide important information about child 
well-being, they have some limitations. First, all of them are based on mothers’ reports, which 
may differ from teachers’ or children’s reports or from direct assessments of the children.'^ Sec- 
ond, the outcomes provide only a snapshot of particular domains of children’s development. For 
example, children’s problem behavior (such as their expressions of anxiety, depression, or ag- 
gression) and positive behavior (such as their interaction with peers and others) are not captured, 
and it may be that the NEWWS programs are most likely to affect these behaviors. Measures col- 
lected in the Child Outcomes Study described in Chapter 12 address these limitations. 

Third, similar measures were collected and constructed for each child regardless of age, 
yet these measures have different implications by child age. For example, partly because of age 
requirements for employment, an adolescent who repeats a grade may be much more likely to 
drop out of school and possibly enter the labor force than a younger school-age child who repeats 
a grade. Dropping out of school is highly correlated with future labor force participation.*^ In ad- 
dition, control group levels on these outcomes might differ by age group: Control group levels of 
suspension or expulsion are naturally higher for adolescents than for early school-age children, 
making it harder for programs to produce any statistically significant changes. 



Mothers with children under age 3, or as young as age 1 in some sites, were exempt from participation in man- 
datory services. These mothers were also excluded from being part of NEWWS. However, because the child age 
groups were created based on information gathered at the five-year survey point, it is possible that some survey re- 
spondents provided information about infants (for example, an infant could have joined the household after random 
assigmnent through marriage or foster care). Consequently, though the majority of the toddlers group is composed of 
children aged 1 and 2 at the time of random assignment, 4.7 percent of these children were under age 1 . 

'^Detailed information about a subset of children aged 3 to 5 in the Child Outcomes Study includes child out- 
comes as evaluated by the mother, a teacher, and the child. See the discussion in Chapter 12 about ways in which 
mothers’ and children’s reports may differ. The New Chance and New Hope evaluations also found that mothers’ 
reports of children’s behavior and academic performance differed from teachers’ reports of these outcomes (Quint, 
Bos, and Polit, 1997; Bos et al., 1999). 

'‘'Freeman and Blanchflower, 1999. 
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To provide some basis for evaluating the magnitude of impacts, Tables 1 1 .2-1 1 .6 and Ta- 
bles 12.1-12.5 report effect sizes in the last column; The accompanying text box describes effect 
sizes in more detail. 



Effect Sizes and the Magnitude of Effects on Child Outcomes 

Evaluating the effects of welfare-to-work programs on child outcomes also requires an as- 
sessment of whether the effects are big or small. An impact may be statistically significant, but is 
it large enough to be deemed important? Evaluating the size of an impact on various measures of 
adult economic outcomes is relatively straightforward. For example, rnost can assess whether or 
not an impact of $200 is a big or small effect on an individual’s annual income. It is much more 
challenging to evaluate whether or not a 10 point change in a scale measuring a child’s behav- 
ioral problems or a 5 percent change in a scale measuring school progress is big or small. 

One method of assessing impact size is to standardize it. To do this, impact estimates can be 
converted into effect sizes. Effect sizes are computed by dividing the impact (the difference in 
outcomes between the program group and the control group) by the standard deviation of the out- 
come for the control group. The value of the effect size provides a standardized measure of the 
program impact that can be used to compare program impacts on outcomes with very different 
scales. Effect sizes generally range from 0 to 1; a larger absolute value indicates a larger impact 
on an outcome and a smaller absolute value indicates a smaller impact. Effect sizes rather than 
percentage changes are reported in the last column of Tables 1 1.2-11.6 and Tables 12.1-12.5. 

How large are these effects? Generally effect sizes of 0.1, 0.3, and 0.5 are considered small, 
medium and large, respectively.® However, these benchmarks are based on nonexperimental stud- 
ies that cover a broad range of topics. One method is to compare effect sizes on adult economic 
outcomes and effects sizes on child outcomes. Most welfare and employment programs generate 
effect sizes of about 0.2 to 0.3 on outcomes such as employment and earnings, and effect sizes on 
child outcomes are generally half this size. Another method is to compare these effect sizes with 
effects produced from child-focused interventions such as the Perry Preschool Program and the 
Abecedarian Project. These child-focused interventions produced effects that generally ranged 
from 0.2 to 1.0. Finally, it is important to consider that even small effects may have a large im- 
pact on the future well-being of a child. Longitudinal studies of children have found that 
achievement and behavior problems can have important implications for children’s well-being as 
adults.*’ For example, achievement and problem behavior in early childhood are related to adoles- 
cent achievement and behavior. Small effects (for example, 0.1 -0.2) may continue to have impli- 
cations for children over their lives. 



“Cohen, 1988; Lipsey, 1990. 

*’Caspi et al., 1998; Masten et al., 1995. 





-229- 



III. The Effect of Welfare-to-Work Approaches on Child Outcomes 

A. Any Child in the Family 



Table 1 1.2 shows impacts on child outcomes for any child in a survey respondent’s fam- 
ily. As mentioned above, these analyses are similar to those conducted at the two-year follow-up 
point. Although some measures are directly comparable with the two-year measures, such as 
grade repetition and suspensions or expulsions, the estimates are shown for the five-year survey 
sample only (rather than for the sample that had information collected at both the two-year and 
five-year points). 

Although there was no consistent pattern of effects on outcomes for any child in a family, 
there were more effects than would be expected by chance. Notably, of the effects that occurred, 
most were produced by the Grand Rapids HCD program. The effects, however, were both favor- 
able and unfavorable within outcome or domain (across program approaches or site). For exam- 
ple, the Atlanta HCD program decreased the proportion of families who had a child retained in 
^ade by approximately 5 percentage points, whereas the Grand Rapids HCD program increased 
it by 5 percentage points. The Grand Rapids HCD program also decreased the proportion of 
families with a child who had a physical, emotional, or mental condition that demanded a lot of 
attention, and the Riverside HCD program increased it. These noted effects also show that effects 
varied by program approach and site (across outcomes or domains). The conclusion drawn here 
about the effects of welfare-to-work programs on these outcomes measured for any child in the 
family are similar to the conclusion drawn at the two-year follow-up point. 

As previously discussed, while evaluating effects on any child in a family may be useful 
in capfiiring a general snapshot, there are reasons to suspect that this kind of analysis may not be 
revealing clear patterns of effects on each child. The next sections review impacts presented in 
Tables 1 1.3-1 1.6 and show that some of the impacts that occurred for the analysis of any child in 
the family (such as effects in Atlanta on grade repetition) also generally occurred, though not al- 
ways statistically significant, across children in many age groups. As would be expected, other 
effects, such as on teen childbearing, were concentrated among adolescents. 

B. Toddlers 

1. Effects on Mothers’ Economic Outcomes 

Impacts for survey sample respondents with toddlers (present in Grand Rapids and Port- 
land samples only) are similar to impacts found for the client survey sample. Thus, mothers of 
these toddlers experienced some increased employment and losses in income from welfare and 
®^rnings. Research finds some evidence that mothers’ employment during the first few years of 
life may be particularly detrimental to cl^ildren’s development.'^ The Canadian Self-Sufficiency 
Project, however, provides some evidence, based on limited outcomes, that welfare and employ- 



'^Baydar and Brooks-Gunn, 1991. 
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National Evaluation of Welfare-to-Work Strategies 
Table 11.2 



Impacts on Child Outcomes During Years 3 to 5 for Any Child in the 
Family at Random Assignment (Aged 6 to 23 at the Five-Year Follow-Up) 



Site and Program 


Sample Program Control Difference 

Size Group (%) Group (%) (Impact) 


Effect 

Size 






Anv child ever repeated a grade 




Atlanta Labor Force Attachment 


1,048 


21.3 


25.6 


1 

* 


-0.10 


Atlanta Human Capital Development 


1,125 


20.8 


25.6 


1 

bo 

* 


-0.11 


Grand Rapids Labor Force Attachment 


1,077 


20.8 


16.9 


3.9 


0.10 


Grand Rapids Human Capital Development 


1,085 


21.9 


16.9 


5.0 ** 


0.13 


Riverside Labor Force Attachment 


1,196 


11.9 


11.6 


0.3 


0.01 


Lacked high school diploma or basic skills 


644 


11.4 


12.6 


-1.2 


-0.03 


Riverside Human Capital Development 


767 


14.9 


12.6 


2.2 


0.06 


Portland 


493 


12.1 


8.2 


3.9 


0.14 






Anv child suspended or expelled 




Atlanta Labor Force Attachment 


1,049 


28.6 


31.9 


-3.3 


-0.07 


Atlanta Human Capital Development 


1,125 


32.4 


31.9 


0.5 


0.01 


Grand Rapids Labor Force Attachment 


1,072 


24.8 


26.5 


-1.7 


-0.04 


Grand Rapids Human Capital Development 


1,081 


22.0 


26.5 


1 

* 


-0.10 


Riverside Labor Force Attachment 


1,190 


21.4 


24.1 


-2.7 


-0.06 


Lacked high school diploma or basic skills 


639 


18.5 


25.0 


-6.5 ** 


-0.15 


Riverside Human Capital Development 


760 


22.7 


25.0 


-2.3 


-0.05 


Portland 


490 


29.4 


29.7 


-0.3 


-0.01 




Anv child ever dropped out of school” 




Atlanta Labor Force Attachment 


1,049 


17.3 


18.4 


-1.1 


-0.03 


Atlanta Human Capital Development 


1,126 


19.6 


18.4 


1.3 


0.03 


Grand Rapids Labor Force Attachment 


1,077 


19.3 


17.8 


1.5 


0.04 


Grand Rapids Human Capital Development 


1,088 


18.6 


17.8 


0.8 


0.02 


Riverside Labor Force Attachment 


1,198 


15.5 


13.7 


1.8 


0.05 


Lacked high school diploma or basic skills 


643 


16.3 


14.2 


2.2 


0.06 


Riverside Human Capital Development 


768 


17.9 


14.2 


3.8 


0.11 


Portland 


495 


21.8 


20.1 


1.6 


0.04 



(continued) 
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Table 11.2 (continued) 



Site and Program 


Sample 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Inroact) 


Effect 

Size 






Any child attended a special class for a 








physicaL emotionah or mental condition^ 




Atlanta Labor Force Attachment 


1,050 


12.6 


12.0 


0.6 


0.02 


Atlanta Human Capital Development 


1,128 


10.5 


12.0 


-1.5 


-0.05 


Grand Rapids Labor Force Attachment 


1,077 


28.9 


28.3 


0.6 


0.01 


Grand Rapids Human Capital Development 


1,084 


28.5 


28.3 


0.2 


0.00 


Riverside Labor Force Attachment 


1,197 


20.1 


18.0 


2.1 


0.06 


Lacked high school diploma or basic skills 


643 


20.9 


18.8 


2.1 


0.05 


Riverside Human Capital Development 


768 


20.3 


18.8 


1.4 


0.04 


Portland 


494 


29.2 


24.8 


4.4 


0.10 




Any child had a ohvsical^ emotional^ or mental condition 




that impeded on mother's ability to eo to work or school^ 


Atlanta Labor Force Attachment 


1,048 


6.9 


5.8 


1.1 


0.05 


Atlanta Human Capital Development 


1,128 


6.9 


5.8 


1.1 


0.05 


Grand Rapids Labor Force Attachment 


1,080 


13.0 


14.7 


-1.6 


-0.05 


Grand Rapids Human Capital Development 


1,091 


10.4 


14.7 


-4.3 ** 


-0.12 


Riverside Labor Force Attachment 


1,200 


10.0 


12.1 


-2.1 


-0.06 


Lacked high school diploma or basic skills 


645 


7.7 


9.9 


-2.2 


-0.07 


Riverside Human Capital Development 


768 


13.9 


9.9 


4.0 ♦ 


0.13 


Portland 


493 


20.6 


19.0 


1.6 


0.04 




Any child had a ohysical. emotional, or mental condition 






that required freauent medical attention^ 




Atlanta Labor Force Attachment 


1,049 


11.7 


10.6 


1.1 


0.04 


Atlanta Human Capital Development 


1,127 


8.9 


10.6 


-1.7 


-0.06 


Grand Rapids Labor Force Attachment 


1,077 


20.3 


19.2 


1.1 


0.03 


Grand Rapids Human Capital Development 


1,085 


15.5 


19.2 


-3.7 


-0.10 


Riverside Labor Force Attachment 


1,198 


13.0 


16.2 


• -3.2 


-0.09 


Lacked high school diploma or basic skills 


645 


11.5 


13.6 


-2.1 


-0.06 


Riverside Human Capital Development 


769 


13.5 


13.6 


-0.1 


0.00 


Portland 


494 


23.3 


20.5 


2.8 


0.07 



(continued) 





Table 11.2 (continued) 



Site and Program 


Sample Program Control Difference 

Size Group (%) Group (%) (Impact) 


Effect 

Size 




Anv child ever had accident iniurv, or ooisoning 




that required an emercencv room visit 




Atlanta Labor Force Attachment 


1,042 


22.4 


25.4 


-3.1 


-0.07 


Atlanta Human Capital Development 


1,118 


21.4 


25.4 


-4.0 


-0.09 


Grand Rapids Labor Force Attachment 


1,071 


35.7 


36.9 


-1.2 


-0.02 


Grand Rapids Human Capital Development 


1,085 


31.5 


36.9 


-5.4 ♦ 


-0.11 


Riverside Labor Force Attachment 


1,186 


38.1 


39.5 


-1.3 


-0.03 


Lacked high school diploma or basic skills 


639 


34.0 


37.2 


-3.2 


-0.07 


Riverside Human Capital Development 


763 


35.9 


37.2 


-1.3 


-0.03 


Portland 


490 


42.1 


41.6 


0.5 


0.01 




Anv child did not live with mother because 






she could not care for child 




Atlanta Labor Force Attachment 


1,051 


4.3 


5.2 


-1.0 


-0.04 


Atlanta Human Capital Development 


1,129 


4.0 


5.2 


-1.3 


-0.06 


Grand Rapids Labor Force Attachment 


1,082 


7.3 


7.2 


0.1 


0.01 


Grand Rapids Human Capital Development 


1,094 


7.7 


7.2 


0.5 


0.02 


Riverside Labor Force Attachment 


1,203 


10.9 


8.6 


2.3 


0.08 


Lacked high school diploma or basic skills 


647 


10.3 


7.0 


3.3 


0.12 


Riverside Human Capital Development 


770 


9.0 


7.0 


2.0 


0.07 


Portland 


495 


11.6 


12.5 


-1.0 


-0.03 






Anv child ever had a babv as a teen^ 




Atlanta Labor Force Attachment 


1,047 


14.8 


19.3 


-4.5 ♦♦ 


-0.12 


Atlanta Human Capital Development 


1,126 


18.2 


19.3 


-1.1 


-0.03 


Grand Rapids Labor Force Attachment 


1,081 


12.5 


15.1 


-2.6 


-0.07 


Grand Rapids Human Capital Development 


1,091 


15.0 


15.1 


-0.1 


0.00 


Riverside Labor Force Attachment 


1,199 


12.0 


10.0 


2.0 


0.07 


Lacked high school diploma or basic skills 


645 


15.5 


10.6 


^ Q ♦♦ 


0.16 


Riverside Human Capital Development 


768 


13.3 


10.6 


2.7 


0.09 


Portland 


495 


9.6 


13.6 


-4.1 


-0.12 



(continued) 
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Table 11.2 (continued) 



SOURCE: MDRC calculations from the Five-Year Client Survey. 

NOTES: See Appendix A.2. 

Owing to missing values, san^le sizes may vary. 

^Measures whether the child dropped.out of school at any point during the child's lifetime. 

**Refers to conditions that were current at the time the survey was administered. 

‘^Measures whether the child had a baby while a teenager at any point during the five-year follow-up 
period. 
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ment programs that increase employment and income do not cause harm to infants and toddlers. 
Less is known about the sole effect of decreased income.'® 

2. Effects on Child Outcomes 

Table 1 1.3 shows outcomes and impacts for children who were toddlers at study entry in 
the Grand Rapids and Portland sites. (These children were roughly 1®‘ and 2"® graders at the time 
of the five-year follow-up.) Approximately 10 percent of these children in the Grand Rapids con- 
trol group and 4 percent in the Portland control group repeated a grade during the last three years 
of follow-up. This range is comparable to national figures, which show that 7 percent of all 2"® 
graders and 10 percent of 2"® graders below poverty were retained in kindergarten or 1®' grade in 
1996.'^ Incidences of suspensions and expulsions among this age group — approximately 6 per- 
cent — seem quite high and, given that many of these children were aged 3 to 5 three years prior 
to the five year follow-up, may reflect suspensions or expulsions from child care arrangements 
rather than school. Possibly more alarming for this age group is that approximately 5 to 6 percent 
of these children in the control group did not live with their mother at some point during the fol- 
low-up period because she could not care for them. 

In general, few program effects were found for this age group. Given that the mothers of 
these toddlers generally experienced increased employment, it is noteworthy that more unfavor- 
able effects were not found. Some of the unfavorable effects of employment on children may be 
influenced by the type and hours of employment, whether or not a mother wants to be employed, 
and the quality of child care. In fact, both Grand Rapids programs produced a consistent (though 
not always statistically significant) pattern of favorable effects for six of the seven outcome 
measures examined. The LFA and HCD program effects on suspensions and expulsions and the 
LFA effect on a condition requiring frequent medical attention were statistically significant and 
of a modest-to-large size relative to effects on child outcomes observed in comparable experi- 
mental studies.'^ 

C. Preschool- Age Children 

1. Effects on Mothers’ Economic Outcomes 

With a few exceptions, impacts on economic outcomes for survey sample respondents 
with preschool-age children are similar in Atlanta, Grand Rapids, and Portland to impacts noted 
for the client survey sample.'^ However, in Riverside impacts are larger and more pronounced for 
survey sample respondents with preschool-age children than for the client survey sample. In par- 
ticular, the Riverside LFA employment impacts for each of the five years of follow-up and cumu- 
lative earnings impacts for survey sample respondents with preschool-age children are nearly 



'^Morris and Michalopoulos, 2000. 

'’U.S. Department of Education, 2001. 

'*Some examples include MFIP (Gennetian and Miller, 2000), SSP (Morris and Michalopoulos, 2000), and FTP 
(Bloom et al., 2000a). 

'^Compared with the client survey sample, the Grand Rapids LFA program decreased earnings (though not 
significant) in year 5 and the Grand Rapids HCD program had no employment effect during the first year of follow- 
up. 
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National Evaluation of Welfare-to-Work Strategies 

Table 11.3 



Impacts on Child Outcomes During Years 3 to 5 for Toddlers 
at Random Assignment (Aged 6 and 7 at the Five-Year Follow-Up) 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Effect 

Size 






Eyer repeated a erade 




Grand Rapids Labor Force Attachment 


361 


8.7 


10.4 


-1.6 


-0.05 


Grand Rapids Human Capital Development 


381 


8.2 


10.4 


-2.1 


-0.07 


Portland 


217 


6.2 


3.6 


2.6 


0.19 






Eyer suspended or expelled 




Grand Rapids Labor Force Attachment 


360 


0.0 


6.2 


-6.4 *** 


-0.27 


Grand Rapids Human Capital Development 


380 


1.6 


6.2 


-4.6 ** 


-0.19 


Portland 


217 


6.6 


6.0 


0.6 


0.03 






Attended a special class for physical. 








emotional, or mental condition^ 




Grand Rapids Labor Force Attachment 


358 


11.5 


15.2 


-3.7 


-0.10 


Grand Rapids Human Capital Development 


379 


14.7 


15.2 


-0.5 


-0.01 


Portland 


215 


20.0 


17.2 


2.8 


0.08 




Had a ohysicaL 


emotional, < 


or mental condition that 






impeded on mother's ability to eo to work or school' 


1 


Grand Rapids Labor Force Attachment 


364 


6.3 


7.6 


-1.3 


-0.05 


Grand Rapids Human Capital Development 


385 


4.8 


7.6 


-2.8 


-0.10 


Portland 


218 


13.2 


12.9 


0.2 


0.01 




Had a physicah emotionaU or mental condition 








that reauired freauent medical attention" 




Grand Rapids Labor Force Attachment 


363 


7.3 


14.1 


-6.8 ** 


-0.20 


Grand Rapids Human Capital Development 


385 


10.3 


14.1 


-3.9 


-0.12 


Portland 


218 


14.4 


11.7 


2.7 


0.09 
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Table 11 .3 (continued) 





Sample 


Program Control Difference 


Effect 


Site and Program 


Size 


Group (%) Group (%) 


(Impact) 


Size 






Ever had accident injury., 


or poisoning that 








required an emergency room yisit 




Grand Rapids Labor Force Attachment 


358 


23.0 24.3 


-1.3 


-0.03 


Grand Rapids Human Capital Development 


380 


19.4 24.3 


-4.9 


-0.11 


Portland 


216 


20.3 27.0 


-6.8 


-0.15 






Did not liye with mother because she 








could not care for child 




Grand Rapids Labor Force Attachment 


364 


6.2 4.6 


1.6 


0.07 


Grand Rapids Human Capital Development 


386 


5.0 4.6 


0.4 


0.02 


Portland 


218 


8.4 6.0 


2.4 


0.10 



SOURCE: MDRC calculations from the Five-Year Client Survey. 



NOTES: See Appendix A.2. 

Standard errors have been adjusted to account for the presence of multiple siblings within a family. 
Owing to missing values, sairple sizes may vary. 

“Refers to conditions that were current at the time the survey was administered. 
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one-third to twice as big as impacts for the client survey sample.^® The effects of mothers’ em- 
ployment for this age group may be either favorable (for example, through role modeling) or un- 
favorable. Although most of the children in this age group are likely to be in school for a large 
portion of the day, they still require supervision during off-school hours, and, thus, the effects of 
mothers’ employment may also depend on the quality of child care or out-of-school arrange- 
ments. Recent evidence suggests that programs that increased employment and increased income 
have positive benefits for young school-age children, particularly in their cognitive develop- 
ment.^' Research on the effects of poverty also finds that the negative effects of poverty are par- 
ticularly pronounced for this age group of children.^^ 

2. Effects on Child Outcomes 

Outcomes and impacts for children of preschool-age at study entry (young school-age at 
the time of the five-year follow-up) are shown in Table 1 1.4. The table shows that 7 to 12 percent 
of children in the control group repeated a grade and 5 to 15 percent were ever suspended or ex- 
pelled during the last three years of the follow-up period. The youngest of these children were 
likely in 2"*' grade. For the oldest of these children, national figures for a roughly comparable age 
group show that 3.3 percent of 4"’ to 8"’ graders were retained in grade in 1995.^^ Roughly 5 to 7 
percent of these children did not live with their mother because she could not care for them. 

Again, few effects were found for this age group. The effects that were found were con- 
centrated in Atlanta and Riverside, were all unfavorable in Atlanta, and, with one exception, fa- 
vorable in Riverside. For example, both Atlanta programs increased the likelihood that a child 
had a condition requiring medical attention and Atlanta LFA increased the likelihood that a child 
had a condition that demanded a lot of attention. The Riverside LFA program decreased grade 
repetition by 5.2 percentage points. Attention should be drawn to the fact that the Riverside LFA 
program increased the likelihood that preschool-age children did not live with their mother be- 
cause she could not care for them. It is interesting that the Riverside LFA program decreased 
grade repetition and, at the same time, increased the likelihood of not living with a parent. 

D. Youu2 School-Age Children 

1. Effects on Mothers’ Economic Outcomes 

Impacts for survey sample respondents with young school-age children are similar to im- 
pacts noted for the client survey sample. As mentioned above for preschool-age children, young 
school-age children may also benefit from or be harmed by mothers’ employment. Because most 
of the children in this age group are likely to be in school for a large portion of the day, the ef- 
fects of mothers’ employment depend on the quality of child care or other arrangements for su- 
pervision during off-school hours. 



^®Some, though not all, of this difference is due to lower control group levels in the sample of respondents with a 
preschool-age child at study entry. 

^'Morris et al., 2001. 

^^Duncan and Brooks-Gurm, 1997. 

^^U.S. Department of Education, 2001. 
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National Evaluation of Welfare-to-Work Strategies 
Table 11.4 



Impacts on Child Outcomes During Years 3 to 5 for Preschool- Age Children 
at Random Assignment (Aged 8 to 10 at the Five-Year FoIIow-Up) 



Site and Program 


Sample 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Effect 
: Size 






Ever repeated a 2 rade 




Atlanta Labor Force Attachment 


766 


11.9 


11.9 


0.0 


0.00 


Atlanta Human Capital Development 


876 


10.2 


11.9 


-1.7 


-0.05 


Grand Rapids Labor Force Attachment 


562 


13.5 


10.4 


3.1 


0.10 


Grand Rapids Human Capital Development 


534 


15.7 


10.4 


5.2 * 


0.17 


Riverside Labor Force Attachment 


829 


4.4 


9.7 


-5.2 *** 


-0.18 


Lacked high school diploma or basic skills 


501 


4.5 


11.4 


-6.9 *** 


-0.22 


Riverside Human Capital Development 


648 


8.9 


11.4 


-2.5 


-0.08 


Portland 


262 


6.4 


7.1 


-0.7 


-0.03 






Ever suspended or expelled 




Atlanta Labor Force Attachment 


766 


8.8 


8.7 


0.2 


0.01 


Atlanta Human Capital Development 


876 


8.8 


8.7 


0.1 


0.00 


Grand Rapids Labor Force Attachment 


561 


6.6 


8.2 


-1.6 


-0.06 


Grand Rapids Human Capital Development 


532 


7.6 


8.2 


-0.6 


-0.02 


Riverside Labor Force Attachment 


820 


4.3 


6.2 


-1.9 


-0.08 


Lacked high school diploma or basic skills 


496 


4.0 


4.7 


-0.8 


-0.04 


Riverside Human Capital Development 


640 


5.5 


4.7 


0.8 


0.04 


Portland 


257 


9.4 


14.8 


-5.4 


-0.16 






Attended a special class for phvsicak 








emotional 


> or mental condition^ 




Atlanta Labor Force Attachment 


766 


8.2 


7.2 


1.0 


0.04 


Atlanta Human Capital Development 


876 


6.1 


7.2 


-1.2 


-0.04 


Grand Rapids Labor Force Attachment 


559 


21.7 


18.9 


2.8 


0.07 


Grand Rapids Human Capital Development 


529 


22.6 


18.9 


3.7 


0.09 


Riverside Labor Force Attachment 


828 


14.5 


13.7 


0.9 


0.03 


Lacked high school diploma or basic skills 


498 


17.2 


15.6 


1.6 


0.04 


Riverside Human Capital Development 


646 


13.0 


15.6 


-2.7 


-0.07 


Portland 


259 


19.5 


18.8 


0.7 


0.02 
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Table 11.4 (continued) 



Site and Program 


Sample Program Control Difference 

Size Group (%) Group (%) (Impact) 


Effect 

Size 




Had a ohysicaK emotionah or mental condition that 




impeded on mother's ability to so to work or school^ 


Atlanta Labor Force Attachment 


767 


5.3 


1.8 


3.5 *** 


0.26 


Atlanta Human Capital Development 


877 


3.2 


1.8 


1.3 


0.10 


Grand Rapids Labor Force Attachment 


564 


7.1 


6.7 


0.3 


0.01 


Grand Rapids Human Capital Development 


536 


6.8 


6.7 


0.1 


0.00 


Riverside Labor Force Attachment 


832 


4.9 


8.5 


-3.6 ** 


-0.13 


Lacked high school diploma or basic skills 


501 


3.4 


7.0 


-3.6 * 


-0.14 


Riverside Human Capital Development 


648 


7.3 


7.0 


0.3 


0.01 


Portland 


262 


13.8 


16.2 


-2.4 


-0.06 




Had a physicaL emotionaL or mental condition 




that reouired freauent medical attention^ 


1 


Atlanta Labor Force Attachment 


767 


7.1 


3.4 


3.7 ** . 


0.20 


Atlanta Human Capital Development 


877 


6.8 


3.4 


3.4 ** 


0.18 


Grand Rapids Labor Force Attachment 


563 


13.9 


11.2 


2.7 


0.08 


Grand Rapids Human Capital Development 


536 


13.2 


11.2 


2.0 


0.06 


Riverside Labor Force Attachment 


832 


6.9 


9.9 


-3.0 


-0.10 


Lacked high school diploma or basic skills 


501 


6.9 


8.7 


-1.8 


-0.06 


Riverside Human Capital Development 


648 


7.2 


8.7 


-1.5 


-0.05 


Portland 


260 


16.0 


11.0 


5.0 


0.16 




Eyer had accident iniury, or poisoning that 






required an emergency room yisit 




Atlanta Labor Force Attachment 


766 


15.1 


16.0 


-0.9 


-0.02 


Atlanta Human Capital Development 


876 


14.2 


16.0 


-1.9 


-0.05 


Grand Rapids Labor Force Attachment 


561 


22.7 


22.0 


0.7 


0.02 


Grand Rapids Human Capital Development 


531 


18.7 


22.0 


-3.3 


-0.08 


Riverside Labor Force Attachment 


819 


25.0 


24.5 


0.6 


0.01 


Lacked high school diploma or basic skills 


498 


22.9 


19.3 


3.6 


0.09 


Riverside Human Capital Development 


640 


20.2 


19.3 


0.9 


0.02 


Portland 


258 


33.3 


24.7 


8.6 


0.20 
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Table 11.4 (continued) 



Sample Program Control Difference Effect 

Site and Program ^ Size Group (%) Group (%) (Impact) Size 

Did not live with mother because she 
could not care for child 



Atlanta Labor Force Attachment 


767 


3.4 


4.8 


-1.4 


-0.07 


Atlanta Human Capital Development 


877 


3.0 


4.8 


-1.8 


-0.09 


Grand Rapids Labor Force Attachment 


564 


6.9 


4.8 


2.2 


0.10 


Grand Rapids Human Capital Development 


538 


5.1 


4.8 


0.3 


0.01 


Riverside Labor Force Attachment 


831 


9.9 


6.6 


3.3 * 


0.13 


Lacked high school diploma or basic skills 


500 


6.8 


6.6 


0.2 


0.01 


Riverside Human Capital Development 


647 


5.2 


6.6 


-1.5 


-0.06 


Portland 


262 


9.8 


7.3 


2.5 


0.09 



SOURCE: MDRC calculations from the Five-Year Client Survey. 

NOTES: See Appendix A.2. 

Standard errors have been adjusted to account for the presence of multiple siblings within a family. 
Owing to missing values, sample sizes may vary. 

^Refers to conditions that were current at the time the survey was administered. 
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2. Effects on Child Outcomes 



Table 11.5 shows outcomes and impacts for children of young school-age at study entry 
(aged 1 1 to 14 at the five-year follow-up). Approximately 4 to 13 percent of children in the con- 
trol group repeated a grade during the last three years of follow-up and 17 to 25 percent were 
ever suspended or expelled. Dropout rates were negligible (at about 1 percent). National figures 
show that 3.3 percent of 4‘*’ to 8‘*’ graders (as mentioned) and 2.4 percent of 9‘*’ to 12‘*’ graders 
were retained in grade in 1995.^"^ Even though the age groups are not directly comparable it sug- 
gests that NEWWS sample rates of grade retention for this age group are higher than national 
rates. Rates of suspensions or expulsions, on the other hand, seem comparable to state figures. 
According to 1999 data from the National Survey of America’s Families, 24.5 percent of children 
aged 12 to 17 in families below 200 percent of poverty in California and 23.2 percent in Michigan 
were suspended or expelled. 

There are generally more effects than would be expected by chance for outcomes of 
young school-age children. Although effects are not consistent within domain, program ap- 
proach, or site, two interesting general patterns emerge. 

First, both Riverside programs increased the likelihood that young school-age children 
did not live with their mother because she could not care for them. The magnitude of this effect 
is also relatively large — almost doubling the likelihood (and effect sizes of 0.15 to 0.25). Both 
Riverside programs decreased suspensions and expulsions for young school-age children (though 
these effects are not supported by other measures that might be expected to be affected by a sus- 
pension or expulsion such as grade repetition or dropping out of school). It appears that the Riv- 
erside programs are producing favorable effects on one measure of academic functioning and, at 
the same time, increasing the likelihood of not living with a parent. One hypothesis is that a 
mother may have decided voluntarily to place her children in an alternative, perhaps temporary, 
living arrangement and this arrangement may have long-term benefits for the child’s develop- 
ment. Or, alternatively, these effects may be capturing two different groups of children: those 
who experienced a decrease in grade repetition and those who were more likely to not live with 
their mother because she could not care for them. 

Second, a pattern of favorable effects occurred for young school-age children in the 
Grand Rapids HCD program for seven of the eight outcome measures examined. These children 
were less likely to be suspended or expelled, less likely to drop out of school (approached statis- 
tical significance at p = .14), less likely to have attended a special class (approached statistical 
significance at p = .14), less likely to have had a condition that demands a lot of attention, and 
less likely to have had a condition that demands frequent medical attention.^^ 



^“*11.8. Department of Education, 2001. 

third interesting pattern emerged in Portland. Portland increased suspensions and expulsions and ever at- 
tending a special class among young school-age children (though not statistically significant at conventional levels). 
These differences are of a similar magnitude, small to medium, as has been found in comparable experimental stud- 
ies (0.2 to 0.3 effect size). The sample size of young school-age children in Portland was relatively small. 
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National Evaluation of Welfare-to-Work Strategies 
Table 11.5 



Impacts on Child Outcomes During Years 3 to 5 for Young School>Age Children 
at Random Assignment (Aged 11 to 14 at the Five-Year Follow-Up) 



Site and Program 


Sanple Program Control Difference 

Size Group (%) Group (%) (Impact) 


Effect 

Size 






Ever repeated a srade 




Atlanta Labor Force Attachment 


652 


11.3 


13.3 


-2.0 


-0.06 


Atlanta Hitman Capital Development 


697 


9.7 


13.3 


-3.6 


-0.10 


Grand Rapids Labor Force Attachment 


524 


14.4 


9.9 


4.4 


0.14 


Grand Rapids Human Capital Development 


513 


12.6 


9.9 


2.7 


0.09 


Riverside Labor Force Attachment 


758 


7.6 


6.0 


1.5 


0.06 


Lacked high school diploma or basic skills 


427 


8.3 


5.0 


3.3 


0.16 


Riverside Human Capital Development 


490 


7.2 


5.0 


2.3 


0.11 


Portland 


265 


4.9 


4.4 


0.5 


0.03 






Ever suspended or expelled 




Atlanta Labor Force Attachment 


653 


25.0 


25.0 


0.1 


0.00 


Atlanta Hitman Capital Development 


694 


26.0 


25.0 


1.1 


0.02 


Grand Rapids Labor Force Attachment 


517 


25.7 


18.7 


7.0 ♦ 


0.18 


Grand Rapids Human Capital Development 


503 


16.7 


18.7 


-1.9 


-0.05 


Riverside Labor Force Attachment 


753 


13.9 


19.5 


-5.6 ♦♦ 


-0.14 


Lacked high school diploma or basic skills 


422 


11.7 


19.6 


-7.9 ♦♦ 


-0.19 


Riverside Human Capital Development 


484 


11.2 


19.6 


-8.4 ♦♦♦ 


-0.21 


Portland 


262 


24.6 


17.1 


7.5 


0.20 






Ever dropped out of school^ 




Atlanta Labor Force Attachment 


0 


0.0 


0.0 


0.0 


0.00 


Atlanta Human Capital Development 


0 


0.0 


0.0 


0.0 


0.00 


Grand Rapids Labor Force Attachment 


525 


0.4 


1.1 


-0.7 


-0.08 


Grand Rapids Human Capital Development 


512 


0.0 


1.1 


-1.2 


-0.14 


Riverside Labor Force Attachment 


760 


0.7 


1.5 


-0.8 


-0.06 


Lacked high school diploma or basic skills 


427 


1.3 


1.1 


0.2 


0.02 


Riverside Human Capital Development 


490 


0.6 


1.1 


-0.5 


-0.05 


Portland 


267 


0.8 


0.0 


0.9 


0.00 



(continued) 
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Table 11.5 (continued) 



Site and Program 


Sample 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Effect 

Size 






Attended a special class for physical 








emotional or mental condition^ 




Atlanta Labor Force Attachment 


654 


7.9 


9.1 


-1.2 


-0.04 


Atlanta Human Capital Development 


699 


6.4 


9.1 


-2.6 


-0.09 


Grand Rapids Labor Force Attachment 


522 


23.1 


25.8 


-2.7 


-0.06 


Grand Rapids Human Capital Development 


507 


19.6 


25.8 


-6.2 


-0.14 


Riverside Labor Force Attachment 


755 


13.2 


13.5 


-0.3 


-0.01 


Lacked high school diploma or basic skills 


423 


12.6 


14.3 


-1.8 


-0.05 


Riverside Human Capital Development 


486 


14.2 


14.3 


-0.2 


-0.01 


Portland 


264 


24.1 


14.4 


9.7 


0.27 




Had a phvsicaK 


emotional. 


or mental condition that 




impeded on mother's abilitv to so to work or school^ 


Atlanta Labor Force Attachment 


655 


4.4 


3.3 


1.0 


0.06 


Atlanta Human Capital Development 


700 


4.4 


3.3 


1.1 


0.06 


Grand Rapids Labor Force Attachment 


529 


10.9 


12.7 


-1.9 


-0.06 


Grand Rapids Human Capital Development 


518 


6.3 


12.7 


-6.4 ** 


-0.19 


Riverside Labor Force Attachment 


761 


7.5 


7.7 


-0.3 


-0.01 


Lacked high school diploma or basic skills 


428 


5.4 


7.1 


-1.8 


-0.07 


Riverside Human Capital Development 


490 


6.4 


7.1 


-0.7 


-0.03 


Portland 


267 


14.0 


12.9 


1.1 


0.03 




Had a physical 


, emotional 


, or mental condition 








that required frequent medical attention^ 




Atlanta Labor Force Attachment 


654 


6.9 


6.6 


0.4 


0.02 


Atlanta Human Capital Development 


700 


4.4 


6.6 


-2.2 


-0.09 


Grand Rapids Labor Force Attachment 


529 


13.7 


15.1 


-1.4 


-0.04 


Grand Rapids Human Capital Development 


518 


9.7 


15.1 


-5.4 * 


-0.16 


Riverside Labor Force Attachment 


761 


8.9 


8.0 


0.9 


0.03 


Lacked high school diploma or basic skills 


428 


6.0 


5.7 


0.3 


0.01 


Riverside Human Capital Development 


490 


7.2 


5.7 


1.5 


0.06 


Portland 


267 


16.3 


14.8 


1.5 


0.04 
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Table 11.5 (continued) 



Site and Program 


San^le 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Effect 

Size 




Ever had accident iniurv. or poisonine that 








required an emereencv room visit 




Atlanta Labor Force Attachment 


652 


14.2 


14.2 


0.0 


0.00 


Atlanta Human Capital Development 


696 


11.4 


14.2 


-2.9 


-0.08 


Grand Rapids Labor Force Attachment 


524 


18.4 


22.4 


-3.9 


-0.09 


Grand Rapids Human Capital Development 


514 


18.2 


22.4 


-4.2 


-0.10 


Riverside Labor Force Attachment 


754 


22.9 


21.8 


1.1 


0.03 


Lacked high school diploma or basic skills 


426 


23.0 


18.8 


4.2 


0.11 


Riverside Human Capital Development 


486 


12.8 


18.8 


-6.0 * 


-0.15 


Portland 


262 


24.3 


24.1 


0.2 


0.00 






Did not live with mother because she 








could not care for child 




Atlanta Labor Force Attachment 


655 


4.6 


3.9 


0.7 


0.04 


Atlanta Human Capital Development 


699 


3.6 


3.9 


-0.3 


-0.02 


Grand Rapids Labor Force Attachment 


529 


6.3 


6.9 


-0.6 


-0.03 


Grand Rapids Human Capital Development 


518 


6.5 


6.9 


-0.4 


-0.02 


Riverside Labor Force Attachment 


759 


10.8 


6.8 


4.0 * 


0.15 


Lacked high school diploma or basic skills 


428 


9.5 


4.2 


5.3 *♦ 


0.25 


Riverside Human Capital Development 


490 


9.4 


4.2 


5.2 ** 


0.24 


Portland 


267 


10.7 


11.9 


-1.2 


-0.04 



SOURCE: MDRC calculations from the Five-Year Client Survey. 

NOTES: See Appendix A. 2. 

Standard errors have been adjusted to account for the presence of multiple siblings within a family. 
Owing to missing values, sanq)le sizes may vary. 

“Measures whether the child dropped out of school at any point during the child's lifetime. 

*^efers to conditions that were current at the time the survey was administered. 



ERIC 



-245- 



2G0 



E. Adolescents 



1. Effects on Mothers’ Economic Outcomes 

Impacts are smaller and more negative in Atlanta and Grand Rapids, similar in Riverside, 
and more positive in Portland for survey sample respondents of adolescents (aged 10 or over at 
time of study entry), than for the client survey sample.^^ The effect of mothers’ employment may 
be particularly pronounced for adolescents by, for example, providing a positive role model and 
encouraging adolescents to prepare for the labor force as young adults. In addition, adolescents in 
low-income families may take on additional responsibilities at home, such as chores, or may en- 
gage in their own employment to help support their family. These activities could have either 
positive or negative consequences for adolescent outcomes. Finally, adolescents might be harmed 
by mothers’ increased employment because it may translate to lack of supervision during a time 
when many may initiate risk-taking behaviors. On the other hand, supervised and high-quality 
out-of-school programs may have particularly beneficial effects for adolescents.^^ 

2. Effects on Child Outcomes 

Outcomes and impacts for children who were adolescents at study entry (aged 1 5 to 23 at 
the five-year follow-up), are shown in Table 11.6. Rates of grade retention during the last three 
years of follow-up varied considerably across sites among adolescents in the control group, from 
4 percent in Portland and Riverside to 17 percent in Atlanta. Dropout rates also varied considera- 
bly, from 17 percent in Riverside to 31 percent in Portland. These dropout rates are higher than 
the dropout rates of lO'*’ to 12'’’ graders for a national sample of 15-to -24-year olds in families at 
the bottom 20 percent of income levels.^^ Approximately 1 5 to 23 percent of adolescents in the 
control group were ever suspended or expelled during the last three years of follow-up. Unsur- 
prisingly, these rates of grade retention, suspensions and expulsions, and dropping out are con- 
siderably higher for adolescents than for younger children. 

One important outcome for adolescents, especially for female adolescents, is teen child- 
bearing, which is correlated with a decreased likelihood of completing schooling and of succeed- 
ing in the labor market and an increased likelihood of receiving public assistance. Furthermore, 
being raised by a teen mother may have negative consequences on children’s development.^^ Ap- 
proximately 13 to 21 percent of adolescents in the NEWWS sample ever had a baby as a teen.^® 
These rates are slightly lower when the sample of adolescents is restricted to those aged 10 or 
over at random assignment but under age 19 at the five-year follow-up (7 to 15 percent; not 
shown). These are double the national rates, though much of the difference is likely due to the 



^*Compared with the client survey sample, survey sample members with adolescents had a negative pattern of 
employment effects, small to no significant earnings gains and a negative though not significant effect on cumulative 
combined income in the Atlanta LFA program; smaller and no significant effect on cumulative earnings in the At- 
lanta HCD program; smaller cumulated earnings effects and larger decreased cumulative combined income in the 
Grand Rapids LFA program; significant decreased cumulative combined income in the Grand Rapids HCD program; 
and slightly larger and more positive employment effects in Portland. 

”Petit et al., 1999; Posner and Vandell, 1994. 

^*U.S. Department of Education, 2001. 

^^Moore et al., 1993. 

^'’Xhis measure includes both males and females. 





National Evaluation of Welfare-to-Work Strategies 
Table 11.6 



Impacts on Child Outcomes During Years 3 to 5 for Adolescents 
at Random Assignment (Aged 15 to 23 at the Five-Year Follow-Up) 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Effect 

Size 






Ever repeated a grade 




Atlanta Labor Force Attachment 


834 


15.2 


17.2 


-1.9 


-0.05 


Atlanta Human Capital Development 


938 


15.5 


17.2 


-1.6 


-0.04 


Grand Rapids Labor Force Attachment 


890 


11.7 


7.5 


4.2 ** 


0.16 


Grand Rapids Human Capital Development 


919 


11.8 


7.5 


4.3 ** 


0.16 


Riverside Labor Force Attachment 


1,126 


7.1 


4.2 


2 9 ♦♦ 


0.14 


Lacked high school diploma or basic skills 


638 


7.0 


4.3 


2.8 


0.13 


Riverside Human Capital Development 


657 


8.2 


4.3 


3.9 ** 


0.18 


Portland 


406 


5.6 


4.4 


1.2 


0.06 






Ever suspended or expelled 




Atlanta Labor Force Attachment 


836 


16.3 


23.2 


-6.8 ** 


-0.16 


Atlanta Human Capital Development 


938 


21.3 


23.2 


-1.9 


-0.05 


Grand Rapids Labor Force Attachment 


891 


21.3 


20.0 


1.3 


0.03 


Grand Rapids Human Capital Development 


924 


18.2 


20.0 


-1.8 


-0.05 


Riverside Labor Force Attachment 


1,120 


15.4 


15.0 


0.3 


0.01 


Lacked high school diploma or basic skills 


634 


13.3 


16.6 


-3.3 


-0.09 


Riverside Human Capital Development 


652 


17.5 


16.6 


0.9 


0.02 


Portland 


395 


16.9 


18.6 


-1.7 


-0.04 






Ever dropped out of school^ 




Atlanta Labor Force Attachment 


836 


22.8 


24.1 


-1.3 


-0.03 


Atlanta Human Capital Development 


937 


26.1 


24.1 


2.0 


0.05 


Grand Rapids Labor Force Attachment 


899 


29.5 


26.0 


3.5 


0.08 


Grand Rapids Human Capital Development 


934 


29.1 


26.0 


3.1 


0.07 


Riverside Labor Force Attachment 


1,122 


18.5 


17.9 


0.6 


0.01 


Lacked high school diploma or basic skills 


635 


18.2 


17.3 


0.9 


0.02 


Riverside Human Capital Development 


657 


22.8 


17.3 


5.4 ♦ 


0.14 


Portland 


409 


36.4 


31.2 


5.2 


0.11 
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Table 11.6 (continued) 



Site and Program 


Sample Program Control Difference 

Size Group (%) Group (%) (Impact) 


Effect 

Size 






Attended a special class for physical. 








emotional, or mental condition'’ 




Atlanta Labor Force Attachment 


835 


5.0 


4.4 


0.6 


0.04 


Atlanta Human Capital Development 


939 


3.9 


4.4 


-0.4 


-0.02 


Grand Rapids Labor Force Attachment 


894 


11.5 


8.5 


3.0 


0.10 


Grand Rapids Human Capital Development 


928 


11.0 


8.5 


2.5 


0.09 


Riverside Labor Force Attachment 


1,128 


7.3 


5.0 


2.2 


0.10 


Lacked high school diploma or basic skills 


640 


7.5. 


3.9 


3.6 * 


0.18 


Riverside Human Capital Development 


660 


6.9 


3.9 


3.0 * 


0.15 


Portland 


407 


8.0 


6.4 


1.6 


0.06 




Had a DhvsicaU emotionaU or mental condition that 




impeded on mother’s ability to go to work or school^ 


Atlanta Labor Force Attachment 


836 


2.3 


3.0 


-0.7 


-0.04 


Atlanta Human Capital Development 


939 


2.9 


3.0 


-0.1 


0.00 


Grand Rapids Labor Force Attachment 


900 


5.7 


3.9 


1.8 


0.09 


Grand Rapids Human Capital Development 


943 


4.3 


3.9 


0.4 


0.02 


Riverside Labor Force Attachment 


1,133 


3.4 


3.2 


0.2 


0.01 


Lacked high school diploma or basic skills 


644 


2.8 


3.1 


-0.2 


-0.01 


Riverside Human Capital Development 


663 


6.2 


3.1 


3.1 * 


0.17 


Portland 


409 


4.5 


2.6 


1.8 


0.10 




Had a physicaU emotional, or mental condition 




that reauired freauent medical attention^ 


Atlanta Labor Force Attachment 


835 


5.3 


4.6 


0.6 


0.03 


Atlanta Human Capital Development 


939 


3.8 


4.6 


-0.8 


-0.04 


Grand Rapids Labor Force Attachment 


899 


6.6 


4.3 


2.3 


0.11 


Grand Rapids Human Capital Development 


941 


4.7 


4.3 


0.4 


0.02 


Riverside Labor Force Attachment 


1,134 


3.4 


6.3 


-2.8 ** 


-0.12 


Lacked high school diploma or basic skills 


644 


2.8 


5.4 


-2.6 * 


-0.11 


Riverside Human Capital Development 


664 


3.8 


5.4 


-1.6 


-0.07 


Portland 


409 


4.5 


4.9 


-0.4 


-0.02 



(continued) 
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Table 11.6 (continued) 



Site and Program 


Sample Program Control Difference 

Size Group (%) Group (%) (Impact) 


Effect 

Size 




Ever had accident, iniurv. or poisonins that 






reauired an emereencv room visit 




Atlanta Labor Force Attachment 


822 


10.8 


12.0 


-1.2 


-0.04 


Atlanta Human Capital Development 


921 


11.8 


12.0 


-0.2 


-0.01 


Grand Rapids Labor Force Attachment 


873 


14.1 


17.4 


-3.2 


-0.09 


Grand Rapids Human Capital Development 


910 


17.3 


17.4 


-0.1 


0.00 


Riverside Labor Force Attachment 


1,104 


15.1 


16.9 


-1.8 


-0.05 


Lacked high school diploma or basic skills 


628 


10.8 


13.3 


-2.5 


-0.07 


Riverside Human Capital Development 


648 


15.9 


13.3 


2.6 


0.08 


Portland 


397 


23.5 


20.1 


3.4 


0.09 






Did not live with mother because 








she could not care for child 




Atlanta Labor Force Attachment 


836 


4.3 


3.9 


0.4 


0.02 


Atlanta Human Capital Development 


937 


4.1 


3.9 


0.2 


0.01 


Grand Rapids Labor Force Attachment 


900 


3.8 


4.6 


-0.9 


-0.04 


Grand Rapids Human Capital Development 


943 


6.4 


4.6 . 


1.7 


0.08 


Riverside Labor Force Attachment 


1,134 


8.1 


5.4 


2.7 * . 


0.12 


Lacked high school diploma or basic skills 


644 


7.7 


4.8 


2.9 


0.13 


Riverside Human Capital Development 


664 


3.3 


4.8 


-1.5 


-0.07 


Portland 


409 


7.9 


10.1 


-2.2 


-0.08 






Ever had a babv as a teen^ 




Atlanta Labor Force Attachment 


829 


16.9 


21.3 


-4.5 


-0.11 


Atlanta Human Capital Development 


936 


22.3 


21.3 


0.9 


0.02 


Grand Rapids Labor Force Attachment 


893 


19.6 


19.7 


-0.2 


0.00 


Grand Rapids Human Capital Development 


923 


20.6 


19.7 


0.8 


0.02 


Riverside Labor Force Attachment 


1,123 


15.4 


13.0 


2.4 


0.07 


. Lacked high school diploma or basic skills 


637 


19.4 


13.2 


6.2 ♦♦ 


0.18 


Riverside Human Capital Development 


658 


16.9 


13.2 


3.7 


0.11 


Portland 


399 


13.8 


17.2 


-3.4 


-0.10 



(continued) 
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Table 11.6 (continued) 



SOURCE: MDRC calculations from the Five-Year Client Survey. 

NOTES: See Appendix A.2. 

Standard errors have been adjusted to account for the presence of multiple siblings within a family. 

Owing to missing values, san^le sizes may vary. 

“Measures whether the child dropped out of school at any point during the child's lifetime. 

^’Refers to conditions that were current at the time the survey was administered. 

^Measures whether the child had a baby while a teenager at any point during the five-year follow-up period. 



n 1 , 

fact that the national figures are not restricted to a low-income or welfare sample. The national 
birth rate in 1998 for teens aged 15 to 19 was 5 percent.^^ In 1997, teen birth rates for females 
aged 15 to 19 were 5.7 percent in California, 6.7 percent in Georgia, 4.4 percent in Michigan, and 
4.7 percent in Oregon.^^ 

The welfare-to-work programs examined in this chapter produced the most effects on 
outcomes for adolescents. The effects were generally unfavorable in the Grand Rapids and River- 
side programs, especially the Riverside HCD program. Both programs in Grand Rapids and both 
programs in Riverside increased grade repetition by 3 percentage points (Riverside LFA) to 4 
percentage points (Grand Rapids HCD). These unfavorable impacts are of concern because chil- 
dren who repeat a grade in high school may be more likely to drop out of school and, as previ- 
ously noted, completed education is highly correlated with future labor force participation. 

In addition to increasing grade repetition, the Riverside HCD program increased the like- 
lihood that an adolescent would drop out of school, increased the percentage of adolescents who 
attended a special class because of a physical, emotional, or mental condition, and increased the 
percentage who had a physical, emotional, or mental condition that demanded a lot of the re- 
spondents’ time. It also increased the percentage of adolescents who were teen parents, an effect 
that approached statistical significance (p = .18). 

Similar unfavorable effects were not found in Atlanta or in Portland. In fact, the Atlanta 
LFA program decreased suspensions or expulsions by almost 7 percentage points and decreased 
teenage childbearing by nearly 5 percentage points (with the latter effect approaching statistical 
significance at p = .1 1). 

Comparisons of NEWWS effects with effects found in other recent studies are discussed 
in the accompanying text box. 

IV. Links Between Effects on Child Outcomes and Program 
Practices or Particular Effects 

The earlier sections began with hypotheses about how targeted outcomes of welfare-to- 
work programs, such as income and employment, may affect child well-being, and then dis- 
cussed impacts on child outcomes. This section expands this discussion and hypothesizes ways in 
which actual program practices or program effects may have led to effects on child outcomes. A 
complete analysis of all the ways that a program may have affected or generated changes in child 
outcomes is beyond the scope of this chapter. However, comparing or “lining up” impacts on 
outcomes presented and discussed in earlier chapters with impacts on child outcomes can reveal 
potential effects on children. With this in mind, this section additionally discusses impacts on 
several other outcomes, including characteristics of employment, household composition, and 
home ownership for mothers of these children (not shown). 



^'Some of the discrepancy could also be due to the inclusion of teen births to males and females in the NEWWS 
sample. 

^^Brown et al., 1999. 

^^Child Trends, December 2000. 
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Comparing NEWWS Adolescent Findings . 

With Those of Other Recent Studies 

Recently released studies have documented some unfavorable effects of welfare and em- 
ployment policy on the outcomes for adolescents. It was found that the Canadian Self- 
Sufficiency Project (SSP), a program that increased full-time employment and income, had no 
effect on major delinquency or academic functioning outcomes but did increase minor delin- 
quency and tobacco, alcohol, and drug use among adolescents.^ Florida’s Family Transition 
Program (FTP), one of the first that evaluated the effects of a time limit, also found some scat- 
tered evidence that adolescents in the program group fared more poorly on a couple of school- 
ing outcomes than adolescents in the control group.*’ Furthermore, additional analyses of data 
from New Hope, a program in Milwaukee that offered an earnings supplement and more gener- 
ous child care and health care benefits for full-time low-income workers, found that New Hope 
had unfavorable effects on some measures of academic functioning.' The Minnesota Family 
Investment Program also produced negative effects on academic functioning outcomes of chil- 
dren aged 10 or over at study entry of recent applicants, although this pattern was generally not 
found for adolescents of long-term welfare recipients.** 

Three of these studies (FTP, New Hope, and SSP) examined adolescents who were ap- 
proximately aged 9 to 15 at the time of study entry, and under age 19 at the time of interview. 
To draw a more precise comparison of the effects of the welfare-to-work programs examined in 
this report with these former studies, impacts on adolescents were rerun for a similar age co- 
hort. In general, the unfavorable effects found for the full adolescent sample in Grand Rapids 
and Riverside were also found for the restricted sample of adolescents. However, notably, as 
was the case for the full sample, no effects were found on these outcomes for the adolescent 
samples in Atlanta and Portland. 

Although confidence in these emerging findings could be bolstered by better and broader 
measures of adolescent development and larger samples, they do provide some consistent evi- 
dence that welfare and employment programs may negatively affect some adolescent chil- 
dren. Why these unfavorable effects are occurring in some sites and programs but not in others, 
in some domains of development but not in others, and whether or not the observed unfavor- 
able effects will result in long-term difficulties as adolescents move into adulthood are espe- 
cially important issues for further research. 



“Morris and Michalopoulos, 2000. 
*’Bloom et al., 2000a. 

‘Bos and Vargas, 2001. 

**Gennetian and Miller, 2000. 




- 252 - 






A. Comparing LFA and HCD Approaches 

Fewer differences than would be expected by chance were found in the effects on child 
outcomes of the LFA program approach compared with the HCD program approach for each of 
the child age groups.^'^ This suggests that children, ranging in age from toddlers to adolescents, 
were not affected differently by education-focused programs than by employment-focused pro- 
grams for the outcomes examined in this chapter. 

B. Toddlers 

Both Grand Rapids programs increased mothers’ employment but decreased average in- 
come from earnings and welfare benefits for mothers of toddlers at study entry. The general pat- 
tern of impacts on toddlers suggests that the children in these families were faring as well as, if 
not doing better than, their control group counterparts. A review of impacts on characteristics of 
employment, income, and child care assistance suggests some ways in which toddlers may have 
been affected. This review suggests that the reduced hours of work and, possibly, the use of 
higher-quality child care, and not decreased income, among program group members compared 
with control group members in Grand Rapids may have contributed to the pattern of effects on 
outcomes for toddlers. 

A closer look at the impacts on hours of employment for these mothers shows that the 
Grand Rapid’s programs increased part-time employment (and decreased full-time employment) 
during the most recent or current job. In fact, fewer hours worked may be what contributed to 
lower income for these mothers. They were also significantly more likely to be in jobs with rotat- 
ing hours and to use child care after leaving welfare because of earnings. In contrast, the Portland 
program increased both full-time and part-time employment and similar effects on child care use 
were not found. Field notes also suggest that caseworkers in Grand Rapids had more leeway in 
administering reimbursement of child care costs (that is, payments were allocated in advance or 
retroactively), and one adult education center that operated the program’s job clubs as well as 
providing some educational activities used by NEWWS sample members provided on-site child 
care. These differences may have influenced the quality of care used (for example, if on-site child 
care was of higher quality). Neither of these circumstances existed in Portland. Finally, there was 



^'’Although the number of significant differences between HCD and LFA outcome levels did not exceed chance 
for each of the child age groups, two interesting patterns did emerge for young school-age children and adolescents. 
For young school-age children, the Grand Rapids HCD outcome levels were lower than LFA outcome levels for sus- 
pensions and the likelihood of having a physical, emotional, or mental condition that demanded a lot of mothers’ 
time; and the Riverside HCD outcome levels were lower than LFA outcome levels for emergency room visits. For 
adolescents, the Atlanta LFA outcome levels were lower than HCD outcome levels for suspensions and the likeli- 
hood of having a baby while a teen; and the Riverside LFA outcome levels were lower than HCD outcome levels for 
emergency room visits and the likelihood of having a physical, emotional, or mental condition that demanded a lot of 
mothers’ time. These patterns are interesting in that they suggest that the type of activity that a mother is first re- 
quired to participate in may have influences on child outcomes that vary by age of the child. For example, an educa- 
tion-focused programs may give mothers more flexibility to better manage their time between participation require- 
ments and their children’s need for supervision than an employment-focused program. In contrast, adolescents may 
benefit more from having a mother engaged in an employment-focused program than an education-focused program 
because of the role modeling that mothers in full-time employment provide. 
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a general pattern of decreased income in both Grand Rapids programs and the Portland program, 
suggesting that decreased income, on its own, is not a likely way in which toddlers were affected. 

C. Adolescents 

Both programs in Riverside produced larger employment effects for mothers of adoles- 
cents than programs in the other sites. As mentioned above, the Riverside programs also pro- 
duced unfavorable effects on the academic outcomes of adolescents. One possible explanation is 
that as mothers’ employment increased, especially full-time employment, adolescents were less 
likely to be supervised, giving them more freedom to engage in risk-taking behaviors. Closer in- 
spection of the employment impacts shows that the Grand Rapids and Riverside programs simi- 
larly had larger employment effects during the first year of the follow-up than the Atlanta and 
Portland programs. Unfortunately, actual measures of supervision were not collected in the sur- 
vey. Nonetheless, unfavorable academic outcomes during the last few years of the follow-up pe- 
riod may be associated with the extent of mothers’ employment early in the follow-up period. 

A review of effects on other important family outcomes such as income and family struc- 
ture suggests that there are other possible reasons why adolescents fared poorly on academic out- 
comes because of these welfare-to-work programs, especially since nearly all of the programs 
increased employment but did not produce similarly unfavorable effects on adolescent outcomes. 
Mothers in the Grand Rapids and Riverside programs also experienced decreased income from 
earnings and welfare benefits during the last three years of the follow-up period. Adolescents 
may have taken on more responsibility contributing to household resources, increasing their own 
employment in response to having less income in the household. Employment during adoles- 
cence, particularly if it is more than 20 hours per week, is associated with difficulty in school.^^ 
In addition, and interestingly, the two programs that produced the most unfavorable effects on 
adolescents — the Grand Rapids LFA program and the Riverside HCD program — also in- 
creased the likelihood that these adolescents’ mothers were married and living with a spouse at 
the five-year follow-up point (not shown). Adolescent children are especially vulnerable to fam- 
ily changes, such as separation, divorce, and marriage.^® 

V. Conclusions 



This chapter examined the effects of these welfare-to-work programs on child outcomes. 
In general few, if any, effects were found. Most program effects did not vary by program ap- 
proach. Employment-focused programs and education-focused programs generally generated 
similar effects. Program effects generally were not concentrated by site, even with the caveat that 
differences between program and control groups were not as distinct by the final year of follow- 
up in Atlanta and Grand Rapids. Portland, in particular, produced few effects, though this was 
not completely surprising since impacts on employment and other economic outcomes were less 
positive for the client survey sample than for the full impact sample. Atlanta appeared to produce 
fewer unfavorable effects than the other sites, especially Grand Rapids and Riverside. 



^^Mortimer et al., 1996, and Steinberg and Dombusch, 1991. 
^^McLanahan, 1997. 





The effects found were primarily clustered by age. First, with data on few outcomes in 
two sites, these welfare-to-work programs produced no unfavorable effects on the outcomes of 
children who were toddlers at study entry (aged 6 and 7 at the five-year follow-up). The lack of 
more unfavorable effects for toddlers is somewhat contrary to what has been found in nonex- 
perimental research that suggests mothers’ employment during the first few years of a child’s life 
produces unfavorable results. Fewer hours of employment, part-time work versus full-time work, 
or the quality of child care arrangements may partially explain why. Second, as has been found in 
the effects of other experimental welfare and employment policies, these welfare-to-work pro- 
grams produced unfavorable effects on the outcomes of children who were adolescents at study 
entry (aged 15 to 23 at the five-year follow-up), especially on academic functioning. The unfa- 
vorable effects found for adolescents may be associated with lack of supervision, decreased in- 
come in the household, or changes in family composition. In any case, it is the well-being of 
these children that perhaps should be more closely monitored when mothers are required to par- 
ticipate in welfare-to-work programs. 



Chapter 12 

Impacts on Young Children 



Chapter 1 1 presented impacts on a circumscribed set of outcomes for children oi all ages 
in the client survey sample (that is, all children in seven programs in four sites). This chapter ex- 
amines in greater depth program impacts for a subset of these children — namely, “focal” chil- 
dren in the Atlanta, Grand Rapids, and Riverside sites who were aged 3 to 5 at baseline (aged 8 
to 10 at the five-year follow-up). These young children and their families constitute the Child 
Outcomes Study (COS) sample (see Chapter 2 for details). ' 

Findings reported here should be viewed as extending those presented for “preschool-age 
children” in Chapter 1 1 , which focused on problem outcomes for children in two developmental 
domains (academic functioning and health and safety). This chapter examines both problem and 
positive outcomes for young children, measured in three developmental domains (academic func- 
tioning, social skills and behavior, and health and safety). In addition, more numerous outcomes 
in each domain are examined and include information from mothers, teachers, and the children 
themselves, as well as from a standardized achievement test. 

Because this chapter focuses on impacts for a subset of the children discussed in Chap- 
ter 11, it is important to begin with a brief overview of key adult impacts for this sample to set 
the stage for interpreting impacts on young children. Earlier chapters presented economic im- 
pacts for the full NEWWS sample (11 programs in seven sites); detailed information on eco- 
nomic impacts for the COS sample and for the client survey sample can be found in Appendix 
I. In sum, despite some differences in these samples,^ the program impacts on economic func- 
tioning for COS families were similar to those for the larger samples. With the exception of a 
decrease in income resulting from the Riverside HCD program in the full sample (compared 
with a similar size, though non-statistically significant, increase in income in the COS sample), 
any differences in economic impacts were in the magnitude, rather than the direction, of a 
given impact. It should be kept in mind that, as discussed in Chapter 1, some control group 
members in the Atlanta and Grand Rapids sites became eligible for program services prior to 
the end of the five-year follow-up period. As a result, in these two sites impacts measured as of 
the end of the follow-up period are probably understated relative to what may have occurred if 
treatment differences had been maintained in those sites. 

This chapter follows a discussion of analysis issues with an examination in depth of pro- 
gram impacts on focal children at the five-year point. (Key patterns of impacts by level of disad- 



Although focal children were aged 8 and 10 at the five-year follow-up, they are characterized as “young” chil- 
dren because they were preschool-age at random assignment. 

COS fanulies consisted solely of single mothers with preschool-age children at random assignment; the larger 
samples included married couples and families not necessarily with a preschool-age child but with children of any 
age. COS mothers were somewhat younger, less likely to have been married, more likely to have a high school di- 
ploma or GED, and they had a slightly higher number of children on average than parents in the full evaluation sam- 
ple. In addition, in Atlanta and Riverside, COS mothers were less likely to have ever worked full time for six months 
or more for the same employer at baseline. See Hamilton, 2000, p. 1 1 . 
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vantage and by focal child gender are noted in footnotes.)^ In an effort to begin to understand the 
ways in which impacts on young children may have come about, the chapter “maps” impacts 
found for focal children onto the economic impacts found for their mothers. It concludes with a 
discussion of findings. (Table 12.1 summarizes impacts on focal child outcomes.) 

I. Key Findings 

• Few impacts on young children were found across sites. However, there 
were more impacts than would be expected by chance. Given that the 
JOBS program was not aimed at children, it is perhaps not surprising that rela- 
tively few impacts were found. It may be that impacts on outcomes important 
to children — such as mothers’ employment, family income, and/or the chil- 
dren’s immediate environments (home, school, child care) — were too few, 
occurred for too brief a period, or were of an insufficient magnitude to lead to 
numerous ihipacts on young children. 

• Within specific domains or programs, patterns of impacts were found. 

• All six programs affected young children’s social skills and behavior, 
with the direction of impacts differing across sites. Both programs in 
Atlanta improved children’s behavior, whereas both programs in Grand 
Rapids worseried children’s behavior. The Riverside LFA program wors- 
ened children’s behavior in the full sample, though for children whose 
mothers lacked a high school diploma or basic skills at study entry, both 
programs in Riverside improved behavior. 

• There were few impacts on young children’s math or reading skills or 
on measures of academic progress and placement (for example, grade 
repetition, performing above or below grade level in reading or math). 

• There were few impacts on measures of health. When impacts were 
found, however, they were all unfavorable. Specifically, both Atlanta 
programs increased the likelihood of mothers’ reporting that focal children 
had had a physical, mental, or emotional condition that required frequent 
medical attention and increased the likelihood of mothers’ reports that 
such a condition impeded on their ability to go to work or school. In addi- 
tion, mothers in the Grand Rapids LFA program and the Riverside HCD 
program were less likely than their control group counterparts to rate focal 
children’s overall health as “very good” or “excellent.” 

• Impacts on young children generally did not vary according to the wel- 
fare-to-work strategy employed. The pattern of impacts was similar for the 
LFA and HCD prograihs in each site. 



more detailed analysis of program impacts on boys and girls at both the two-year and the five-year follow-up 
is beyond the scope of this report and will be done as part of future analyses. 
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National Evaluation of Welfare-to-Work Strategies 
Table 12.1 



Summaty of Impacts on Focal Child Outcomes 





Atlanta 


Grand Rapids ' 




Riverside 




LFA 


HCD 


LFA 


HCD 


LFA 

Full 


LFA 

In-need® 


HCD 


Social skills and 


6F 


5F 








2F 


2F 


behavior 




2 f 








2f 


1 f 


(19 measures) 






2U 


6U 


3U 








1 u 




1 u 


1 u 


1 u 






Academic functioning 
















Behavioral 




2F 






1 F 






adjustment 


1 f 














to school 






1 U 


2U 


1 U 






(5 measures) 
















Achievement 






IF 




J 


1 F 




(6 measures) 
















Academic progress 


1 F 


1 F 






1 F 


1 F 




and placement 














1 f 


( 1 1 measures) 










1 U 








l" 








1 u 


lu 




Attendance 


1 U 


1 u 










lU 


(2 measures) 


1 u 














Health and safety 


2U 


1 u 


1 u 








lU 


(5 measures) 




1 u 












Other 
















(1 measure) 

















NOTES: ”F" indicates a statistically significant favorable inpact. "U" indicates a statistically significant 
unfavorable impact, "f ' indicates a favorable inpact above the cutoff for statistical significance but part of the 
overall pattern, "u” indicates an unfavorable inpact above the cutoff for statistical significance but part of the 
overall pattern. See Chapter 2 for the definition of a pattern. Blank spaces indicate that there were no impacts. 

*This sample is a subset of the full Riverside LFA sanple, containing only those members who lacked a high 
school diploma or basic skills at random assignment. 

*"The Atlanta LFA program decreased the percentage of children identified by teachers as needing and 
receiving special services, which would be favorable if the program decreased the percentage of children needing 
services but unfavorable if the program decreased the percentage of children in need of services who received them. 
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II. Analysis Issues 



This section provides an overview of the outcomes examined for young children in the 
COS and discusses issues relating to interpreting information on child outcome measures from 
multiple sources. (A more detailed description of measures, including internal consistency reli- 
abilities, can be found in Appendix J.) 

A. Child Outcomes Examined 

Child outcomes in three developmental domains were measured: social skills and behav- 
ior, academic functioning, and health and safety. Measures tapping both positive and problem 
outcomes were examined. Outcomes in the social skills and behavior domain consisted of sub- 
scales adapted from the Social Skills Ratings System, which was designed to tap “social behav- 
iors that can affect teacher-student relations, peer acceptance, and academic performance.”^ 
Among the subscales are those that measure children’s positive skills and behaviors (such as co- 
operation and self-control) as well as problem behaviors (“externalizing” behaviors such as fight- 
ing and arguing, “internalizing” behaviors such as acting sad or depressed, and hyperactive be- 
haviors such as acting impulsively and being disruptive). 

Outcomes in the academic functioning domain are particularly diverse and include (1) 
measures related to behavioral adjustment to school (for example, school engagement, discipli- 
nary problems), (2) scores on a standardized assessment of focal children’s math and reading 
skills (the Woodcock-Johnson Tests of Achievement — Revised),^ administered by trained inter- 
viewers, (3) measures related to academic progress and placement (for example, grade repetition, 
performance below grade level), and (4) measures of attendance (absenteeism, tardiness). 

Outcomes in the health and safety domain include a global rating of focal children’s gen- 
eral health; the percentage of mothers rating focal children’s health as very good or excellent; the 
percentage of mothers reporting that focal children had a physical, mental, or emotional condi- 
tion that required frequent medical attention; the percentage of mothers reporting that focal chil- 
dren had a physical, mental, or emotional condition that impeded on their going to work or 
school; and the percentage of mothers reporting that focal children had an accident, injury, or 
poisoning requiring an emergency room visit. 

An additional outcome pertains to focal children’s living arrangements: the percentage of 
mothers reporting that focal children had lived away from them at some point since random as- 
signment because they could not care for them. This measure is not a child outcome in the sense 
that it reflects directly on the developmental status or well-being of children. However, given the 
potential effect on children of living apart from their mother and a concern that welfare reform 
not contribute to this outcome, it is reported on here (under the category “other”). 

The accompanying chart summarizes the outcomes measured and the source of informa- 
tion for each outcome. Mothers reported on focal children’s social skills and behavior, behavioral 
adjustment to school, academic progress and placement, health and safety, and living 

“'Gresham and Elliot, 1990, p. 1. 

^Woodcock and Johnson, 1989, 1990. 
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Developmental Domains and Focal Child Outcomes Measured 
at the Five-Year FoIIow-Up, by Source 





Assessment 


Mother 


Child 


Teacher 


Social Skills and Behavior 










Externalizing 




X 




X 


Internalizing 




X 




X 


Hyperactivity 




X 




X 


Cooperation 




X 


X 


X 


Positive assertion 




X 


X 


X 


Self-control 




X 


X 


X 


Empathy 






X 




Responsibility 




X 






Interpersonal skills 








X 


Positive approaches to learning 








X 


Academic Functioning 










Behavioral adjustment to school 










School engagement 






X 


X 


Suspended or expelled 




X 




X 


Disciplinary action 








X 


Achievement 










Math skills 


X 








Above-average math 


X 








Below-average math 


X 








Reading skills 


X 








Above-average reading 


X 








Below-average reading 


X 








Academic progress and placement 










Grade repetition 




X 




X 


Below grade level in math 








X 


Above grade level in math 








X 


Below grade level in reading 








X 


Above grade level in reading 








X 


In remedial math group 








X 


In remedial reading group 








X 


Physical, mental, or emotional 










condition requiring a special 










class or school 


X 








Needs and receives services* 








X 


Needs but does not receive services 








X 


Attendance 










Absent 








X 


Tardy 








X 


Health and Safety 










General health rating 




X 






In very good or excellent health 




X 






Physical, mental, or emotional condition 










requiring frequent medical attention 




X 






Physical, mental, or emotional condition 










impeding mother’s work 




X 






Accident, injury, or poisoning requiring 










an emergency room visit 




X 






Other 










Lived apart from mother 




X 







*See Appendix J for a full description of services examined. 





arrangements. Children reported on their positive social skills and their engagement in school. 
Teachers reported on focal children’s social skills and behavior, behavioral adjustment to school, 
academic progress and placement, and attendance.^ 

B. Multiple Reporters 

Obtaining information from multiple sources can provide a more comprehensive picture 
of children’s behavior and development. Parents are an important source of such information, as 
they are “usually the most knowledgeable about their child’s behavior across time and situa- 
tions.”’ For school-age children, teachers can also provide important information on children’s 
competencies and problems in the school setting. Children’s reports can also be informative, for 
they represent subjective views of their own competencies and shortcomings. Finally, objective 
assessments of children’s academic functioning — such as standardized achievement tests — 
provide a measure of performance on a specific “skill,” free from subjective biases of infor- 

o 

mants. Standardized assessments also are useful for comparing children’s performance with 
those of their peers in the sample as well as to age-mates nationwide. 

Mothers, teachers, and children provided information on children’s social skills and be- 
havior. Survey items in the social skills and behavior domain were worded specifically to capture 
social skills displayed in the classroom (for the teachers’ report measures), social skills displayed 
in the home (for the mothers’ report measures), and social skills more generally, as displayed in 
the home and/or the school (for the children’s report measures). Thus, ratings of children’s social 
skills and behavior by teachers should be seen as reflecting children’s classroom behavior in the 
current school year, ratings of children’s social skills and behavior by mothers likely capture the 
more stable component of children’s behavior in the family, and children’s ratings of their own 
social skills and behavior reflect their perceptions of their behavior in both the home and the 
school settings. Impacts on a social skill as rated by one reporter and not another thus may indi- 
cate changes in context-specific behavior. By contrast, a pattern of impacts that occurs on the 
same measure across reporters suggests a more global impact on children’s positive and/or prob- 
lem behavior. 

Ratings of children’s academic functioning were provided largely by teachers and, thus, 
reflect the teachers’ views of how the children were performing (academically, socially) in 
school. While it is reasonable to expect that these reports — and any impacts on academic meas- 
ures — would not contradict objective assessments of (and any impacts on) children’s math and 
reading skills, such contradictions may arise if, for example, teachers’ evaluations of children’s 
academic capabilities reflect a more global perception of how the children are performing and 
behaving in school. 



Ws noted in Chapter 2, the sample for the analyses of achievement test scores and of the mother- and child- 
reported child outcomes comprises 2,332 mothers and their focal children, and the sample for the analyses of 
teacher-reported child outcomes includes responses from 1,472 teachers of focal children in the final mother and 
child survey sample. 

^Achenbach, 1991. 

^Admittedly, even objective assessments of childrens’ academic skills may contain some cultural biases. 
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Finally, information on focal children’s health and safety was obtained solely from moth- 
ers. Neither objective ratings of physical health (for example, height or weight) nor diagnoses by 
medical professionals were obtained. Thus, it is not clear to what extent mean levels and impacts 
on health and safety measures reflect the focal children’s actual health status and to what extent 
they reflect mothers’ perceptions of the children’s health. 

III. Impacts on Children 

This section presents impacts of the LFA and HCD programs in Atlanta, Grand Rapids, 
and Riverside on children’s developmental outcomes in the domains of social skills and behav- 
ior, academic functioning, health and safety, and the outcome relating to living arrangements. 

A. Social Skills and Behavior 

Table 12.2 shows impacts on outcomes relating to focal children’s social skills and be- 
havior. As described above, outcomes examined in this domain come from reports by mothers, 
teachers, and children of both positive and problem behaviors. 

The largest concentration of impacts across programs, and typically within a given pro- 
gram, occurred in this domain of outcomes. All six programs affected children’s social skills and 
behavior, though the direction of these impacts differed depending on the site (the Atlanta and 
Grand Rapids programs) or on mothers’ level of education at baseline (the Riverside programs). 
Both Atlanta programs improved social skills and behavior, as did the Riverside HCD program 
and the Riverside LFA program for children whose mothers lacked a high school diploma or ba- 
sic skills at baseline. In contrast, both Grand Rapids programs had unfavorable impacts on chil- 
dren’s social skills and behavior, as did the Riverside LFA program for the full sample (that is, 
regardless of mothers’ level of education at baseline). 

Both Atlanta programs decreased problem behaviors and simultaneously increased posi- 
tive behaviors.^ Specifically, both programs decreased teacher-reported levels of externalizing, 
internalizing, and hyperactive behavior and increased teacher-reported levels of interpersonal 
skills. Further, both programs resulted in lower levels of mother-rated externalizing behavior and 
higher levels of teacher-reported positive assertion (a measure of children’s positive initiations 
toward others in social situations), though the differences for the HCD program were just beyond 
the cutoff for statistical significance. The Atlanta HCD program also increased levels of chil- 
dren’s self-control as rated by the teacher.'® 

In contrast to Atlanta, the impacts on social skills and behavior in Grand Rapids were uni- 
formly unfavorable, with both programs decreasing focal children’s positive behaviors and the 
HCD program also increasing problem behaviors. For instance, both programs decreased child- 



’interestingly, the favorable impacts on behavior in the Atlanta programs were largely concentrated among girls, 
with girls showing fewer problem behaviors and more positive behaviors in both programs and, in the HCD program, 
fewer disciplinary problems as well. 

‘°There was a single finding that did not fit with this overall pattern of favorable impacts on social skills in the 
Atlanta programs: The LFA program decreased child-reported cooperation, though this difference was just beyond 
the cutoff for statistical significance (p = . 1 1). 
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Table 12.2 



Impacts on Social Skills and Behavior^ 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 




Externalizine behavior franse of 0 to 18) (mother report) 


Atlanta Labor Force Attachment 


542 


3.8 


4.2 


-0.5 ♦♦ 


-0.20 


Atlanta Human Capital Development 


609 


3.9 


4.2 


-0.3 


-0.13 


Grand Rapids Labor Force Attachment 


376 


5.5 


5.6 


-0.1 


-0.02 


Grand Rapids Human Capital Development 


358 


5.6 


5.7 


-0.1 


-0.02 


Riverside Labor Force Attachment 


454 


4.7 


4.9 


-0.3 


-0.10 


Lacked high school diploma or basic skills 


284 


4.8 


5.0 


-0.2 


-0.09 


Riverside Human Capital Development 


368 


4.6 


5.0 


-0.4 


-0.15 




Externalizing behavior (ran&e of 0 to 15) (teacher report) 


Atlanta Labor Force Attachment 


363 


4.1 


5.1 


-1.0 ♦♦ 


-0.29 


Atlanta Human Capital Development 


409 


4.2 


5.1 


-0.9 ♦♦ 


-0.25 


Grand Rapids Labor Force Attachment 


283 


3.7 


4.0 


-0.2 


-0.08 


Grand Rapids Human Capital Development 


259 


4.8 


3.8 


0.9 ♦ 


0.28 


Riverside Labor Force Attachment 


323 


3.9 


3.1 


0.8 ♦♦ 


0.28 


Lacked high school diploma or basic skills 


205 


3.9 


3.4 


0.5 


0.18 


Riverside Human Capital Development 


272 


3.0 


3.5 


-0.4 


-0.16 




Internalizing behavior (range of 0 to 24) (mother report! 


Atlanta Labor Force Attachment 


546 


7.7 


7.8 


-0.1 


-0.03 


Atlanta Human Capital Development 


617 


7.7 


7.8 


0.0 


-0.01 


Grand Rapids Labor Force Attachment 


376 


8.8 


9.1 


-0.3 


-0.08 


Grand Rapids Human Capital Development 


357 


8.6 


9.1 


-0.5 


-0.15 


Riverside Labor Force Attachment 


456 


8.6 


8.2 


0.4 


0.14 


Lacked high school diploma or basic skills 


286 


8.5 


8.4 


0.1 


0.03 


Riverside Human Capital Development 


370 


8.4 


8.3 


0.0 


0.02 



(continued) 
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Table 12.2 (continued) 



Site and Program 


Sarrqjle 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 




Internalizine behavior Grange of 0 to 18) (teacher reoort) 


Atlanta Labor Force Attachment 


357 


4.0 


5.2 


-1.2 ** 


-0.33 


Atlanta Human Capital Development 


404 


4.2 


5.1 


-0.9 ** 


-0.25 


Grand Rapids Labor Force Attachment 


281 


3.6 


4.0 


-0.4 


-0.13 


Grand Rapids Human Capital Development 


257 


4.8 


3.9 


0.9 * 


0.27 


Riverside Labor Force Attachment 


323 


3.9 


3.2 


p 

bo 

« 

« 


0.27 


Lacked high school diploma or basic skills 


204 


3.9 


3.5 


0.5 


0.17 


Riverside Human Capital Development 


266 


3.1 


3.6 


-0.4 


-0.15 




Hvoeractivitv (range of 0 to 18) (mother report) 




Atlanta Labor Force Attachment 


545 


5.9 


5.9 


0.0 


-0.02 


Atlanta Human Capital Development 


616 


5.9 


5.9 


-0.1 


-0.03 


Grand Rapids Labor Force Attachment 


379 


6.8 


6.9 


-0.1 


-0.02 


Grand Rapids Human Capital Development 


360 


6.8 


6.9 


-0.1 


-0.02 


Riverside Labor Force Attachment 


459 


6.6 


6.1 


0.5 


0.17 


Lacked high school diploma or basic skills 


290 


6.4 


6.1 


0.2 


0.11 


Riverside Human Capital Development 


373 


6.2 


6.1 


0.1 


0.03 




Hvoeractivitv (range of 0 to 6) (teacher reoort) 




Atlanta Labor Force Attachment 


360 


1.9 


2.2 


-0.3 ♦ 


-0.24 


Atlanta Human Capital Development 


401 


1.9 


2.2 


-0.4 ♦♦ 


-0.25 


Grand Rapids Labor Force Attachment 


283 


2.0 


2.0 


0.0 


0.03 


Grand Rapids Human Capital Development 


261 


2.5 


2.0 


0.5 ♦♦ 


0.32 


Riverside Labor Force Attachment 


321 


2.3 


1.9 


0.4 ♦♦ 


0.29 


Lacked high school diploma or basic skills 


204 


2.2 


2.0 


0.2 


0.16 


Riverside Human Capital Development 


270 


2.1 


2.1 


0.0 


-0.01 



(continued) 





Table 12.2 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 






Cooperation (ranee of 0 to 30) (child report) 




Atlanta Labor Force Attachment 


558 


22.5 


23.2 


-0.7 


-0.17 


Atlanta Human Capital Development 


627 


22.9 


23.1 


-0.3 


-0.06 


Grand Rapids Labor Force Attachment 


378 


22.7 


23.4 


-0.6 


-0.14 


Grand Rapids Human Capital Development 


362 


22.3 


23.4 


-1.0 ** 


-0.23 


Riverside Labor Force Attachment 


459 


21.9 


22.0 


-0.1 


-0.01 


Lacked high school diploma or basic skills 


289 


22.2 


21.9 


0.3 


0.07 


Riverside Human Capital Development 


384 


22.2 


21.9 


0.3 


0.07 




Cooperation fran&e of 0 to 39) (mother report) 




Atlanta Labor Force Attachment 


549 


22.4 


21.9 


0.5 


0.10 


Atlanta Human Capital Development 


620 


22.1 


21.9 


0.3 


0.05 


Grand Rapids Labor Force Attachment 


382 


21.3 


22.1 


-0.9 


-0.13 


Grand Rapids Human Capital Development 


363 


21.4 


22.1 


-0.7 


-0.11 


Riverside Labor Force Attachment 


460 


22.3 


22.1 


0.2 


0.03 


Lacked high school diploma or basic skills 


290 


22.5 


22.3 


0.2 


0.05 


Riverside Human Capital Development 


374 


23.0 


22.2 


0.7 


0.15 




Cooperation (ran&e of 0 to 27) (teacher report) 




Atlanta Labor Force Attachment 


367 


16.2 


15.1 


1.0 


0.18 


Atlanta Human Capital Development 


410 


16.2 


15.2 


1.0 


0.17 


Grand Rapids Labor Force Attachment 


286 


14.9 


16.2 


-1.3 * 


-0.22 


Grand Rapids Human Capital Development 


261 


14.8 


16.0 


-1.2 


-0.21 


Riverside Labor Force Attachment 


325 


15.1 


16.2 


-1.1 


-0.19 


Lacked high school diploma or basic skills 


206 


15.4 


15.2 


0.2 


0.03 


Riverside Human Capital Development 


275 


15.1 


15.2 


-0.2 


-0.03 



(continued) 
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Table 12.2 (continued) 



Site and Program 


San^le 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 




Positive assertion (ranee of 0 to 30) (child report) 




Atlanta Labor Force Attachment 


557 


20.0 


20.1 


0.0 


-0.01 


Atlanta Human Capital Development 


629 


19.9 


20.1 


-0.2 


-0.05 


Grand Rapids Labor Force Attachment 


379 


21.3 


22.2 


-0.9 * 


-0.21 


Grand Rapids Human Capital Development 


363 


20.9 


22.2 


-1.3 ** 


-0.29 


Riverside Labor Force Attachment 


458 


19.9 


20.0 


-0.2 


-0.03 


Lacked high school diploma or basic skills 


289 


20.0 


19.5 


0.6 


0.12 


Riverside Human Capital Development 


381 


20.1 


19.6 


0.5 


0.12 




Positive assertion (ranee of 0 to 30) (mother report) 




Atlanta Labor Force Attachment 


549 


20.7 


20.7 


0.0 


-0.01 


Atlanta Human Capital Development 


619 


21.0 


20.7 


0.3 


0.06 


Grand Rapids Labor Force Attachment 


380 


21.2 


21.6 


-0.4 


-0.08 


Grand Rapids Human Capital Development 


360 


21.4 


21.6 


-0.2 


-0.05 


Riverside Labor Force Attachment 


459 


21.5 


21.7 


-0.1 


-0.03 


Lacked high school diploma or basic skills 


289 


20.9 


20.9 


0.0 


0.00 


Riverside Human Capital Development 


371 


21.8 


21.0 


0.8 


0.21 




Positive assertion (ranee of 0 to 18) (teacher report) 




Atlanta Labor Force Attachment 


363 


11.6 


10.8 


0.8 * 


0.25 


Atlanta Human Capital Development 


405 


11.6 


10.9 


0.7 


0.21 


Grand Rapids Labor Force Attachment 


279 


10.6 


11.0 


-0.4 


-0.13 


Grand Rapids Human Capital Development 


253 


10.5 


11.0 


-0.5 


-0.15 


Riverside Labor Force Attachment 


318 


11.8 


11.8 


0.1 


0.02 


Lacked high school diploma or basic skills 


201 


12.2 


11.1 


1.2 ♦♦ 


0.39 


Riverside Human Capital Development 


266 


11.2 


11.3 


0.0 


-0.01 



(continued) 
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Table 12.2 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 






Self-control (ranee of 0 to 30) (child report) 




Atlanta Labor Force Attachment 


558 


19.6 


19.5 


0.1 


0.02 


Atlanta Human Capital Development 


629 


19.3 


19.5 


-0.3 


-0.06 


Grand Rapids Labor Force Attachment 


378 


19.9 


20.6 


-0.7 


-0.15 


Grand Rapids Human Capital Development 


362 


19.4 


20.6 


-1.2 ** 


-0.25 


Riverside Labor Force Attachment 


461 


18.7 


19.0 


-0.3 


-0.06 


Lacked high school diploma or basic skills 


291 


18.9 


18.7 


0.2 


0.05 


Riverside Human Capital Development 


382 


19.0 


18.6 


0.4 


0.08 




Self-control (range of 0 to 30) (mother report) 




Atlanta Labor Force Attachment 


548 


16.1 


16.1 


0.0 


0.00 


Atlanta Human Capital Development 


619 


15.8 


16.0 


-0.3 


-0.07 


Grand Rapids Labor Force Attachment 


380 


15.4 


15.4 


0.0 


0.01 


Grand Rapids Human Capital Development 


359 


15.4 


15.5 


-0.1 


-0.03 


Riverside Labor Force Attachment 


461 


16.4 


15.8 


0.6 


0.14 


Lacked high school diploma or basic skills 


291 


16.2 


15.2 


1.0 * 


0.26 


Riverside Human Capital Development 


373 


16.5 


15.3 


1:2 ** 


0.29 




Self-control (ranee of 0 to 27) (teacher reoort) 




Atlanta Labor Force Attachment 


359 


16.3 


15.4 


0.9 


0.15 


Atlanta Human Capital Development 


405 


16.7 


15.4 


1.3 * 


0.23 


Grand Rapids Labor Force Attachment 


261 


15.9 


16.0 


-0.2 


-0.03 


Grand Rapids Human Capital Development 


239 


15.3 


16.1 


-0.8 


-0.14 


Riverside Labor Force Attachment 


307 


18.0 


17.7 


0.3 


0.05 


Lacked high school diploma or basic skills 


198 


18.4 


17.0 


1.5 


0.28 


Riverside Human Capital Development 


260 


17.3 


17.0 


0.2 


0.05 
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Table 12.2 (continued) 



Site and Program 


Sanple 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 






Empathy (ranee of 0 to 30) (child report) 




Atlanta Labor Force Attachment 


559 


21.2 


20.9 


0.2 


0.05 


Atlanta Human Capital Development 


627 


21.4 


21.0 


0.4 


0.08 


Grand Rapids Labor Force Attachment 


377 


22.3 


22.7 


-0.4 


-0.09 


Grand Rapids Human Capital Development 


361 


22.3 


22.8 


-0.4 


-0.10 


Riverside Labor Force Attachment 


457 


21.0 


21.7 


-0.7 


-0.15 


Lacked high school diploma or basic skills 


288 


21.1 


21.5 


-0.4 


-0.08 


Riverside Human Capital Development 


380 


20.9 


21.6 


-0.7 


-0.14 




ResDonsibilitv (ranee of 0 to 27^ (mother report) 




Atlanta Labor Force Attachment 


559 


17.4 


17.6 


-0.2 


-0.06 


Atlanta Human Capital Development 


624 


18.1 


17.5 


0.5 


0.14 


Grand Rapids Labor Force Attachment 


383 


17.8 


17.7 


0.1 


0.03 


Grand Rapids Human Capital Development 


365 


17.8 


17.8 


0.0 


0.01 


Riverside Labor Force Attachment 


473 


17.6 


17.8 


-0.3 


-0.07 


Lacked high school diploma or basic skills 


298 


17.4 


17.3 


0.1 


0.03 


Riverside Human Capital Development 


390 


18.3 


17.4 


0.9 * 


0.24 




Interoersonal skills (ranee of 0 to 12) (teacher report) 


Atlanta Labor Force Attachment 


364 


7.7 


7.1 


0.6 ** 


0.26 


Atlanta Human Capital Development 


408 


7.9 


7.1 


0.7 ** 


0.30 


Grand Rapids Labor Force Attachment 


279 


7.4 


7.5 


-0.2 


-0.07 


Grand Rapids Human Capital Development 


254 


7.1 


7.5 


-0.5 


-0.18 


Riverside Labor Force Attachment 


322 


8.2 


8.3 


-0.1 


-0.04 


Lacked high school diploma or basic skills 


205 


8.4 


7.9 


0.5 


0.26 


Riverside Human Capital Development 


272 


8.0 


8.0 


-0.1 


-0.03 



(continued) 



Table 12.2 (continued) 



Site and Program • 


Sanple 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 




Positive aoproaches to learning (range of 0 to 18) 
(teacher report) 




Atlanta Labor Force Attachment 


366 


10.9 


10.4 


0.5 


0.16 


Atlanta Human Capital Development 


408 


10.9 


10.4 


0.5 


0.15 


Grand Rapids Labor Force Attachment 


279 


10.1 


10.7 


-0.6 


-0.16 


Grand Rapids Human Capital Development 


254 


10.0 


10.7 


-0.7 


-0.19 


Riverside Labor Force Attachment 


324 


10.6 


10.8 


-0.2 


-0.07 


Lacked high school diploma or basic skills 


206 


10.5 


10.2 


0.3 


0.10 


Riverside Human Capital Development 


274 


10.5 


10.3 


0.2 


0.07 



SOURCE: Child Trends calculations from the Five-Year Child Outcomes Study survey (mother, teacher, and child 
reports). 



NOTES: See Appendix A.2. 

A higher score on each measure indicates that the child demonstrates more of the construct. 
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reported levels of assertion and decreased levels of both child- and teacher-reported cooperation, 
although the differences in child-reported cooperation in the LFA program and teacher-reported 
cooperation in the HCD program were just beyond the cutoff for statistical significance." The 
HCD program also decreased children’s reports of their own self-control and simultaneously in- 
creased their problem behavior, increasing teacher-reported levels of externalizing, internalizing, 
and hyperactive behavior. 

The impacts on social skills and behavior were fewer in Riverside than in Atlanta or in 
the Grand Rapids HCD program and varied according to mothers’ level of education at baseline. 
The Riverside LFA program increased problem behaviors for the full sample (regardless of 
mothers’ level of education at baseline), while both the LFA and the HCD programs increased 
positive behaviors for the subsample of children whose mothers had limited education at base- 
line. Specifically, in the full sample the LFA program increased levels of teacher-reported exter- 
nalizing, internalizing, and hyperactive behavior. This program also increased mother-reported 
levels of hyperactive behavior, though this difference was just beyond the cutoff for statistical 
significance (an effect size of .17). Yet, for the subsample of mothers without a high school di- 
ploma or basic skills at baseline, both the LFA and HCD programs improved behavioral out- 
comes, for example, increasing mother-reported levels of self-control in children. Further, the 

LFA program increased teacher-rated levels of assertion, self-control, and interpersonal skills for 

1 *1 

this subgroup. Notably, the favorable impact of the Riverside LFA program on teacher-reported 
levels of positive assertion for this subgroup was quite large compared with the other impacts in 
the social skills and behavior domain, with an effect size of .39. The Riverside HCD program 
also increased mother-reported levels of responsibility and assertion, although the latter impact 
was just beyond the cutoff for statistical significance. 

With few exceptions, the magnitude of the impacts in the social skills and behavior do- 
main across the six programs ranged from .20 to .30 of a standard deviation. These effect sizes 
fall at the lower end of the effect size range for some of the more successful programs that di- 
rectly targeted children, such as the Perry Preschool Program and the Abecedarian Project." 
Given that the programs evaluated in this chapter did not directly intervene with children, the fact 
that these effect sizes are within the range of those found in the child-focused programs is per- 
haps surprising and suggests that the welfare-to-work programs examined here affected chil- 
dren’s behavior in nontrivial ways. 



' 'Although the difference for teacher-reported cooperation in the Grand Rapids HCD program was just beyond 
the cutoff for statistical significance, it was of a size comparable to similar impacts in other programs that were 
statistically significant. 

'^The impacts on behavior in the Riverside LFA program occurred especially (or, in many cases, only) for the 
least disadvantaged subgroup and were part of a larger picture of unfavorable impacts for this subgroup that ex- 
tended past behavior into teachers’ reports of academic progress and placement. Further, the increases in problem 
behavior in this program were concentrated largely among girls and were accompanied by an increase in disciplinary 
problems among girls. 

'^Though the latter two differences were above the cutoff for statistical significance, they were both of a size 
comparable to impacts in other programs that were statistically significant. 

'‘'Bermeta-Clement et al., 1984; Campbell and Ramey, 1994. 
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Further, the impacts described in this section were found on social skills and behavior 
that are likely to be meaningful for children’s future development. Positive social skills may be 
important for children’s success in the school setting, as well as for the development of positive 
relationships with peers and adults, and therefore may have implications for successful function- 
ing later in life.*^ In addition, research has shown that problem social behaviors that go untreated 
are related to “poor academic performance, and may result in later social maladjustment prob- 
lems or serious psychopathology.”'^ Early problem behaviors have also been identified as predic- 
tors of later delinquency in adolescence.'^ 

B. Academic Functioning 

Table 12.3 shows the impacts on child outcomes related to academic functioning. As 
mentioned, these outcomes were reported by mothers, teachers, and children, as well as obtained 
from a standardized assessment, and fell into the following subdomains: behavioral adjustment to 
school, academic achievement, academic progress and placement, and attendance. 

In general, the impacts on academic functioning were less numerous and their patterns 
less clear than those found within the social skills and behavior domain. Yet in five of the six 
programs there were impacts on measures relating to behavioral adjustment to school that, 
though few, were consistent with the impacts on social skills and behavior noted above. This 
suggests that impacts on more global ratings of children’s social skills and behavior were accom- 
panied by simultaneous changes on measures pertaining specifically to children’s behavioral ad- 
justment to school — namely, disciplinary problems and engagement in school. 

The pattern of impacts on disciplinary problems and school engagement generally follows 
that found in the social skills and behavior domain. Both Atlanta programs decreased the likeli- 
hood that focal children had a discipline problem requiring parental notification, and the Grand 
Rapids HCD program increased this likelihood. The Riverside LFA program also increased prob- 
lems in the full sample, increasing the likelihood of having disciplinary action taken at least 
weekly with the focal child in the few months prior to the survey. These four programs generally 
altered the likelihood of discipline problems by between 9 and 16 percentage points, decreasing 
levels from about 56 to 42 percent in both Atlanta programs and increasing levels from 31 to 47 
percent in the Grand Rapids HCD program and from 30 to 39 percent in the Riverside LFA pro- 
gram. In addition, following the pattern of social skills and behavior impacts, the Atlanta HCD 
program also increased teacher-reported levels of children’s engagement in school, whereas both 
Grand Rapids programs decreased child-reported levels of school engagement. The effect sizes 
of these impacts ranged between .20 and .30. Hence, the general pattern of findings in the social 
skills and behavior domain was replicated and extended for measures of academic functioning 
relating to children’s behavioral adjustment to school. The singular exception to this pattern is a 
decrease in suspensions and expulsions (as reported by mothers) in the Riverside LFA program. 



'^Although relatively little is known about predictors and consequences of positive development in children and 
youth, there are some indications that positive development can lead to more successful future functioning. See 
Moore and Glei, 1995. 

'^Gresham and Elliot, 1990, p. 1. See, for example, Coie and Dodge, 1983; Cowen et al., 1973; Parker and 
Asher, 1987. 

'^Farrington, 1987, as cited in Yoshikawa, 1995. 
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National Evaluation of Welfare-to-Work Strategies 

Table 12.3 



Impacts oil Academic Functioning 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 




School eneaeement (range of 0 to 21) (child report) 




Atlanta Labor Force Attachment 


552 


16.6 


16.8 


-0.2 


-0.07 


Atlanta Human Capital Development 


617 


16.7 


16.8 


-0.1 


-0.05 


Grand Rapids Labor Force Attachment 


377 


16.3 


17.2 


-0.9 ♦♦ 


-03.0 


Grand Rapids Human Capital Development 


358 


16.5 


17.1 


-0.6 ♦ 


-0.21 


Riverside Labor Force Attachment 


459 


16.2 


16.3 


-0.1 


-0.02 


Lacked high school diploma or basic skills 


290 


16.2 


16.0 


0.2 


0.05 


Riverside Human Capital Development 


379 


16.4 


16.1 


0.3 


0.09 




School engagement (range of 0 to 33) (teacher report) 


Atlanta Labor Force Attachment 


364 


19.8 


19.1 


0.7 


0.11 


Atlanta Human Capital Development 


414 


20.6 


19.2 


1.4 ♦ 


0.21 


Grand Rapids Labor Force Attachment 


286 


18.9 


19.2 


-0.3 


-0.04 


Grand Rapids Human Capital Development 


262 


18.2 


19.1 


-0.9 


-0.14 


Riverside Labor Force Attachment 


324 


18.7 


19.5 


-0.9 


-0.13 


Lacked high school diploma or basic skills 


205 


19.0 


18.5 


0.5 


0.08 


Riverside Human Capital Development 


274 


19.0 


18.7 


0.4 


0.06 




Suspended or expelled since last interview ° (%) 










(mother report) 




Atlanta Labor Force Attachment 


599 


8.5 


6.5 


2.0 


0.10 


Atlanta Human Capital Development 


676 


7.0 


6.1 


1.0 


0.05 


Grand Rapids Labor Force Attachment 


428 


5.7 


9.0 


-3.3 


-0.13 


Grand Rapids Human Capital Development 


410 


7.0 


8.3 


-1.3 


-0.05 


Riverside Labor Force Attachment 


533 


2.8 


6.8 


-4.0 ♦ 


-0.17 


Lacked high school diploma or basic skills 


338 


3.3 


4.6 


-1.3 


-0.07 


Riverside Human Capital Development 


438 


6.0 


4.5 


1.4 


0.08 
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Table 12.3 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 




Had disciDlmarv action taken weekly (%) (teacher report) 


Atlanta Labor Force Attachment 


369 


30.8 


37.3 


-6.5 


-0.17 


Atlanta Human Capital Development 


412 


32.1 


36.6 


-4.5 


-0:12 


Grand Rapids Labor Force Attachment 


285 


34.3 


31.6 


2,7 


0.06 


Grand Rapids Human Capital Development 


261 


37.8 


31.2 


6.6 


0.15 


Riverside Labor Force Attachment 


325 


39.4 


29.5 


9.9 * 


0.24 


Lacked high school diploma or basic skills 


206 


32.4 


30.6 


1.8 


0.05 


Riverside Human Capital Development 


274 


33.9 


30.9 


3.0 


0.08 




Had a discipline problem that resulted in parentfs) 






being notifled this school year (%) (teacher report) 




Atlanta Labor Force Attachment 


365 


46,4 


55.3 


-8.9 


-0.22 


Atlanta Human Capital Development 


407 


42.4 


55.8 


-13.4 *** 


-0.34 


Grand Rapids Labor Force Attachment 


284 


33.4 


32.7 


0.7 


0.02 


Grand Rapids Human Capital Development 


259 


47.1 


31.4 


15.7 ** 


0.36 


Riverside Labor Force Attachment 


325 


35.7 


29.2 


6.4 


0.15 


Lacked high school diploma or basic skills 


205 


24.6 


31.9 


-7.3 


-0.19 


Riverside Human Capital Development 


274 


35.0 


33.2 


1.8 


0.05 






Broad Math Score on the 






Woodcock- Johnson Tests of Achieyement-Reyised 




Atlanta Labor Force Attachment 


557 


99.7 


99.5 


0.2 


0.01 


Atlanta Human Capital Development 


627 


101.1 


99.4 


1.7 


0.11 


Grand Rapids Labor Force Attachment 


377 


100.0 


99.2 


0.8 


0.05 


Grand Rapids Human Capital Development 


363 


97.8 


99.0 


-1.2 


-0.08 


Riverside Labor Force Attachment 


463 


98.9 


97.8 


1.1 


0.07 


Lacked high school diploma or basic skills 


292 


97.1 


94.6 


2.5 


0.19 


Riverside Human Capital Development 


383 


95.6 


95.4 


0.2 


0.01 
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Table 12.3 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 






Had above-average Broad Math Score on the 






Woodcock-Johnson Tests of Achievement-Revised (%> 


Atlanta Labor Force Attachment 


557 


30.2 


27.4 


2.8 


0.08 


Atlanta Human Capital Development 


627 


31.3 


27.8 


3.5 


0.10 


Grand Rapids Labor Force Attachment 


377 


31.4 


23.6 


7.8 ♦ 


0.20 


Grand Rapids Human Capital Development 


363 


28.0 


24.2 


3.8 


0.10 


Riverside Labor Force Attachment 


463 


30.8 


24.2 


6.6 


0.16 


Lacked high school diploma or basic skills 


292 


30.7 


15.9 


14.8 ♦♦♦ 


0.49 


Riverside Human Capital Development 


383 


18.6 


17.9 


0.7 


0.02 






Had below-average Broad Math Score on the 






Woodcock-Johnson Tests of Achievement-Revised (Vo) 


Atlanta Labor Force Attachment 


557 


23.5 


25.8 


-2.3 


-0.06 


Atlanta Human Capital Development 


627 


22.2 


26.3 


-4.1 


-0.11 


Grand Rapids Labor Force Attachment 


377 


24.5 


21.4 


3.1 


0.08 


Grand Rapids Human Capital Development 


363 


28.5 


21.8 


6.8 


0.18 


Riverside Labor Force Attachment 


463 


25.2 


27.0 


-1.9 


-0.05 


Lacked high school diploma or basic skills 


292 


27.9 


31.7 


-3.8 


-0.10 


Riverside Human Capital Development 


383 


34.9 


30.1 


4.9 


0.13 






Broad Reading Score on the 






Woodcock-Johnson Tests of Achievement-Revised 




Atlanta Labor Force Attachment 


557 


94.7 


95.0 


-0.2 


-0.02 


Atlanta Human Capital Development 


627 


95.5 


95.0 


0.5 


0.04 


Grand Rapids Labor Force Attachment 


377 


100.2 


98.8 


1.4 


0.09 


Grand Rapids Human Capital Development 


363 


97.7 


98.7 


-1.0 


-0.06 


Riverside Labor Force Attachment 


463 


95.4 


95.2 


0.3 


0.02 


Lacked high school diploma or basic skills 


292 


93.5 


92.6 


0.9 


0.06 


Riverside Human Capital Development 


383 


93.7 


93.2 


0.5 


0.03 



(continued) 




3I9 



-274- 



Table 12.3 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 




Had above-averaee Broad Reading Score on the 
Woodcock-Johnson Tests of Achievement-Revised (%) 


Atlanta Labor Force Attachment 


557 


15.9 


17.5 


-1.6 


-0.05 


Atlanta Human Capital Development 


627 


17.5 


17.6 


-0.1 


0.00 


Grand Rapids Labor Force Attachment 


377 


31.5 


25.4 


6.1 


0.15 


Grand Rapids Human Capital Development 


363 


27.1 


26.1 


l.O 


0.02 


Riverside Labor Force Attachment 


463 


18.9 


19.6 


-0.7 


-0.02 


Lacked high school diploma or basic skills 


292 


17.4 


16.9 


0.5 


0.02 


Riverside Human Capital Development 


383 


15.2 


17.9 


-2.7 


-0.09 




Had below-average Broad Reading Score on the 
Woodcock-Johnson Tests of Achievement-Revised (%) 


Atlanta Labor Force Attachmen.. 


557 


33.3 


36.5 


-3.2 


-0.08 


Atlanta Human Capital Development 


627 


32.1 


36.3 


-4.3 


-0.11 


Grand Rapids Labor Force Attachment 


377 


24.4 


27.9 


-3.5 


-0.08 


Grand Rapids Human Capital Development 


363 


30.2 


28.7 


1.5 


0.04 


Riverside Labor Force Attachment 


463 


36.0 


33.7 


2.2 


0.05 


Lacked high school diploma or basic skills 


292 


43.2 


38.7 


4.5 


0.11 


Riverside Human Capital Development 


383 


40.1 


37.1 


3.0 


0.08 


Atlanta Labor Force Attachment 


599 


Repeated a grade since last interview^ (%) 
(mother report) 

12.8 13.0 -0.1 


0.00 


Atlanta Human Capital Development 


676 


10.9 


12.9 


-2.0 


-0.07 


Grand Rapids Labor Force Attachment 


428 


16.4 


11.8 


4.6 


0.15 


Grand Rapids Human Capital Development 


410 


16.1 


12.2 


3.8 


0.13 


Riverside Labor Force Attachment 


533 


3.2 


9.4 


-6.1 ♦♦ 


-0.23 


Lacked high school diploma or basic skills 


338 


4.8 


10.2 


♦ 

1 


-0.22 


Riverside Human Capital Development 


438 


6.1 


9.9 


-3.8 


-0.16 
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Table 12.3 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 






Ever repeated 


a grade (%) (teacher report) 




Atlanta Labor Force Attachment 


336 


15.0 


11.8 


3.2 


0.13 


Atlanta Human Capital Development 


381 


11.8 


11.9 


-0.1 


0.00 


Grand Rapids Labor Force Attachment 


248 


8.9 


13.2 


-4.3 


-0.14 


Grand Rapids Human Capital Development 


238 


15.7 


12.0 


3.6 


0.12 


Riverside Labor Force Attachment 


288 


3.7 


5.3 


-1.6 


-0.08 


Lacked high school diploma or basic skills 


181 


5.6 


7.0 


-1.4 


-0.07 


Riverside Human Capital Development 


242 


3.7 


6.8 


-3.1 


-0.16 




Below grade level in math (%) (teacher reoort) 




Atlanta Labor Force Attachment 


362 


35.9 


35.7 


0.2 


0.01 


Atlanta Human Capital Development 


401 


32.1 


35.0 


-2.9 


-0.08 


Grand Rapids Labor Force Attachment 


283 


38.9 


38.4 


0.5 


0.01 


Grand Rapids Human Capital Development 


257 


42.7 


38.2 


4.4 


0.10 


Riverside Labor Force Attachment 


321 


47.4 


39.7 


7.8 


0.18 


Lacked high school diploma or basic skills 


204 


46.6 


47.1 


-0.5 


-0.01 


Riverside Human Capital Development 


269 


43.0 


45.0 


-2.0 


-0.05 




Above grade level in math (%) (teacher report) 




Atlanta Labor Force Attachment 


362 


9.8 


8.0 


1.8 


0.08 


Atlanta Human Capital Development 


401 


7.7 


8.3 


-0.6 


-0.03 


Grand Rapids Labor Force Attachment 


283 


8.1 


11.7 


-3.6 


-0.12 


Grand Rapids Human Capital Development 


257 


9.2 


11.9 


-2.7 


-0.09 


Riverside Labor Force Attachment 


321 


6.7 


8.5 


-1.8 


-0.06 


Lacked high school diploma or basic skills 


204 


4.3 


4.8 


-0.5 


-0.03 


Riverside Human Capital Development 


269 


5.2 


5.3 


-0.1 


0.00 
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Table 12.3 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control Difference 
Group (Impact) 


Effect 

Size 




Below erade level in readme (%) (teacher report) 


Atlanta Labor Force Attachment 


369 


38.8 


42.7 


-3.9 


-0.10 


Atlanta Human Capital Development 


412 


39.3 


42.8 


-3.5 


-0.09 


Grand Rapids Labor Force Attachment 


285 


50.3 


44.7 


5.6 


0.12 


Grand Rapids Human Capital Development 


259 


50.9 


45.6 


5.4 


0.12 


Riverside Labor Force Attachment 


326 


49.5 


42.4 


7.1 


0.16 


Lacked high school diploma or basic skills 


207 


56.0 


54.1 


2.0 


0.05 


Riverside Human Capital Development 


274 


50.1 


53.1 


-3.0 


-0.07 




Above crade level in readine (%) (teacher report) 


Atlanta Labor Force Attachment 


369 


12.6 


6.9 


5.6 * 


0.27 


Atlanta Human Capital Development 


412 


11.4 


6.2 


5.2 * 


0.25 


Grand Rapids Labor Force Attachment 


285 


9.6 


12.9 


-3.2 


-0.11 


Grand Rapids Human Capital Development 


259 


14.8 


13.1 


1.7 


0.05 


Riverside Labor Force Attachment 


326 


9.6 


15.6 


-6.0 


-0.17 


Lacked high school diploma or basic skills 


207 


7.5 


9.8 


-2.3 


-0.09 


Riverside Human Capital Development 


274 


9.4 


10.9 


-1.5 


-0.06 




In remedial math eroup (%) (teacher report) 


Atlanta Labor Force Attachment 


368 


27.7 


33.9 


-6.2 


-0.17 


Atlanta Human Capital Development 


411 


31.1 


32.5 


-1.4 


-0.04 


Grand Rapids Labor Force Attachment 


287 


30.7 


28.1 


2.7 


0.06 


Grand Rapids Human Capital Development 


262 


24.7 


28.4 


-3.7 


-0.09 


Riverside Labor Force Attachment 


326 


27.5 


27.5 


0.0 


0.00 


Lacked high school diploma or basic skills 


207 


28.7 


33.5 


-4.7 


-0.13 


Riverside Human Capital Development 


272 


33.2 


32.0 


1.2 


0.03 
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Table 12.3 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 




In remedial readme erouD (%) (teacher report) 




Atlanta Labor Force Attachment 


368 


36.7 


36.7 


0.0 


0.00 


Atlanta Human Capital Development 


413 


35.6 


36.8 


-1.2 


-0.03 


Grand Rapids Labor Force Attachment 


285 


34.1 


39.3 


-5.3 


-0.12 


Grand Rapids Human Capital Development 


262 


41.8 


40.6 


1.2 


0.03 


Riverside Labor Force Attachment 


326 


44.5 


34.8 


9.7 * 


0.23 


Lacked high school diploma or basic skills 


207 


54.3 


42.9 


11.4 


0.28 


Riverside Human Capital Development 


274 


42.8 


43.8 


-0.9 


-0.02 






Goes to a special class or school, or eets 






special help in school, for a physical, emotional, or 








mental condition (%) (mother report) 




Atlanta Labor Force Attachment 


600 


8.2 


6.7 


1.5 


0.07 


Atlanta Human Capital Development 


678 


5.9 


6.7 


-0.8 


-0.04 


Grand Rapids Labor Force Attachment 


428 


21.5 


20.0 


1.4 


0.04 


Grand Rapids Human Capital Development 


410 


23.9 


19.7 


4.2 


0.11 


Riverside Labor Force Attachment 


533 


14.5 


12.2 


2.3 


0.08 


Lacked high school diploma or basic skills 


338 


17.3 


14.1 


3.2 


0.11 


Riverside Human Capital Development 


438 


12.0 


14.2 


-2.2 


-0.08 






Identified as needine and receivine 








special services (%) (teacher report) 




Atlanta Labor Force Attachment 


377 


33.5 


42.9 


-9.4 * 


-0.24 


Atlanta Human Capital Development 


419 


43.0 


42.5 


0.5 


0.01 


Grand Rapids Labor Force Attachment 


288 


44.4 


45.5 


-1.1 


-0.02 


Grand Rapids Human Capital Development 


264 


48.4 


45.8 


2.5 


0.05 


Riverside Labor Force Attachment 


330 


46.7 


44.4 


2.2 


0.05 


Lacked high school diploma or basic skills 


209 


52.3 


54.1 


-1.8 


-0.04 


Riverside Human Capital Development 


279 


50.1 


54.9 


-4.8 


-0.12 
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Table 12.3 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 






Identified as needing and not receiying 








soecial seryices (%) (teacher report) 




Atlanta Labor Force Attachment 


377 


40.7 


38.2 




2.6 


0.07 


Atlanta Human Capital Development 


419 


32.9 


38.3 




-5.4 


-0.14 


Grand Rapids Labor Force Attachment 


288 


33.0 


29.3 




3.7 


0.09 


Grand Rapids Human Capital Development 


264 


31.9 


28.3 




3.6 


0.09 


Riverside Labor Force Attachment 


330 


41.2 


32.0 




9.2 


0.22 


Lacked high school diploma or basic skills 


209 


43.7 


34.2 




9.5 


0.24 


Riverside Human Capital Development 


279 


29.1 


33.2 




-4.1 


-0.11 




Days absent during current school year (%) 








(teacher report) 






Atlanta Labor Force Attachment 


318 


5.3 


4.2 




1.1 


0.23 


Atlanta Human Capital Development 


359 


5.9 


3.8 




2.1 ** 


0.43 


Grand Rapids Labor Force Attachment 


254 


7.2 


6.6 




0.6 


0.07 


Grand Rapids Human Capital Development 


243 


6.1 


6.8 




-0.7 


-0.09 


Riverside Labor Force Attachment 


303 


6.2 


5.6 




0.6 


0.09 


Lacked high school diploma or basic skills 


191 


6.4 


6.1 




0.3 


0.06 


Riverside Human Capital Development 


256 


8.8 


6.3 




2.5 ** 


0.48 




Days tardy during current school year (%) 










(teacher report) 






Atlanta Labor Force Attachment 


316 


4.0 


2.3 




1.7 * 


0.44 


Atlanta Human Capital Development 


360 


2.7 


2.2 




0.5 


0.13 


Grand Rapids Labor Force Attachment 


253 


3.0 


3.9 




-0.9 


-0.13 


Grand Rapids Human Capital Development 


242 


3.9 


3.9 




0.0 


0.00 


Riverside Labor Force Attachment 


301 


4.0 


3.4 




0.6 


0.09 


Lacked high school diploma or basic skills 


190 


3.5 


3.2 




0.3 


0.05 


Riverside Human Capital Development 


253 


3.4 


3.3 




0.1 


0.01 



SOURCE: Child Trends calculations from the Five-Year Child Outcomes Study survey (mother, teacher, and child 
reports). 

NOTES: See Appendix A. 2. 

Woodcock-Johnson Broad Math and Broad Reading scores are age-standardized, with a mean of 100 and a 
standard deviation of 15. 

“Measures of events "since the last interview" apply to years 3 to 5 for 2,163 COS sample members (who 
responded to the survey at 2 years) and years 1 to 5 for 1 69 sample members (who were interviewed at baseline but 
not at 2 years). 



Impacts in the other subdomains of academic functioning were less prevalent across the 
six programs. Impacts on standardized tests of academic achievement were rare, with only two 
impacts found across the six programs and across the six measures. There were no aggregate im- 
pacts on mean age-standardized achievement test scores, with mean scores for program and con- 
trol group children across the six programs falling between 93 and 100 on the reading assessment 
(representing the 32"^ and 50*^ percentiles, respectively) and between 95 and 101 on the math as- 
sessment (representing the 37* and 51®* percentiles, respectively).'^’'^ There were also no impacts 
on the prevalence of “6e/ow-average” scores — that is, mean age-standardized scores less than 
90. The two impacts that were found were both favorable, increasing the prevalence of “above- 
average” scores (scores above 110) on the math assessment in the Grand Rapids LFA program 
and in the Riverside LFA program for the subgroup of children whose mothers had limited edu- 
cation at baseline. 

Within the subdomain of academic progress and placement, impacts were also few, with no 
impacts on five of the eight measures. There were some impacts on measures of whether the focal 
child had repeated a grade (as reported by the mother), was in a remedial reading group, and per- 
formed above grade level in reading. The Riverside LFA program decreased the likelihood of hav- 
ing repeated a grade, dropping to about 4 percent fi’om a control group level of about 10 percent for 
both the full sample and the subgroup of children whose mothers lacked a high school diploma or 
GED at baseline. The Riverside HCD program also decreased grade repetition by about 4 percent- 
age points, though this difference was just above the cutoff for statistical significance. Yet the Riv- 
erside LFA program simultaneously increased the number of children who were in a remedial read- 
ing group by about 10 percentage points for both the full sample and those with limited education 
(fi-om levels of about 35 and 43 percent, respectively), although the difference for the subgroup with 
limited education was just beyond the cutoff for statistical significance.^" The Riverside LFA pro- 



‘®Scores on that fall between 90 and 1 10, representing the 25* to the 75* percentiles, respectively, are classified 
as “average” or “normal” (Woodcock and Mather, 1989, 1990). 

'Nevertheless, there were impacts on mean math scores within gender subgroups in both Riverside programs. 
Both programs increased mean math scores for boys whose mothers lacked a high school diploma or GED at base- 
line. Additionally, the Riverside HCD program also increased boys’ mean reading scores and, interestingly, de- 
creased girls’ mean math and mean reading scores. 

^''While the impacts of the Riverside LFA program may seem inconsistent (simultaneously decreasing the num- 
ber of children repeating a grade and increasing the number in a remedial reading group), these impacts occurred on 
experiences that apply to only a minority of the sample and thus may not pertain to the same children. In other 
words, the children who were in a remedial reading group may not be the same individuals who repeated a grade. 
Alternatively, it may be that teachers’ evaluations of children’s reading and math skills are highly dependent on chil- 
dren’s classroom behavior, in which case the impact on being in a remedial reading group may reflect the behavioral 
impacts of this program more so than its impacts on measures of academic progress or placement. This speculation is 
supported by a set of findings for the least disadvantaged subgroup, in which there is a general pattern of unfavorable 
impacts on teachers’ reports of behavior. At the same time, however, there is a single finding that appears to diverge 
fi:om this pattern. Program group children of the least disadvantaged mothers are more likely to score above average 
on the reading test than controls yet are less likely to be rated as above grade level in reading by their teachers. This 
suggests that teachers’ evaluations of children’s performance in reading incorporate more than the children’s actual 
reading skills (which, in fact, increased for program group children). For example, teachers perceiving less school 
engagement and more disciplinary problems in children (which was more tme for the Grand Rapids HCD program 
group than for the control group) may have difficulty accurately gauging a child’s reading skills if the child is having 
difficulty remaining seated or following instmctions during reading class. Likewise, even among children with iden- 
tical capabilities in reading, a teacher might be less likely to move a child into a higher-level reading group (and/or 
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gram also increased the percentage of children needing but not receiving services (an unfavorable 
finding), a difference just beyond the cutoff for statistical significance. 

Finally, both Atlanta programs increased the proportion of children reported by their teach- 
ers as being above grade level in reading, increasing the levels from about 6 to 12 percent in both 
programs. In addition, the Atlanta LFA program increased the proportion of children needing and 
receiving services (an impact that may be favorable or unfavorable depending on whether it reflects 
more children needing services or more of those who need services receiving them). 

Impacts on measures of attendance were also relatively scarce. Generally, control group 
children were absent for between 4 and 7 percent of days and were tardy between 2 and 4 percent 
of days in the current school year. Three of the programs (Atlanta LFA and HCD and Riverside 
HCD) increased absences, although the difference for the Atlanta LFA program was just beyond 
the cutoff for statistical significance.^' The Atlanta LFA program also increased the percentage of 
days that children were tardy. All four of these differences were in the range of 1 to 3 percentage 
points. As speculated in the next section, the impacts may be related to unfavorable health im- 
pacts that were found in these three programs. 

C. Health and Safety 

Impacts on measures of children’s health and safety are shown in Table 12.4. The out- 
comes in this domain are available only from mothers’ reports. 

The impacts in this domain of development were relatively few. There were no impacts 
on three of the six measures of health and safety, including mean levels of children’s general 
health, as reported by mothers. Overall, the children in these sites seemed to be relatively healthy 
compared with children nationally. The proportion of control group children rated by their moth- 
ers as being in very good or excellent health ranged from 85 percent (in Atlanta) to 90 percent (in 
Grand Rapids). These levels are higher than the 1994 estimates from the National Health Inter- 
view Survey, in which 78.5 percent of all children aged 5 to 17 and 61.5 percent of those from 
families with annual incomes below $10,000 were rated by their parents as being in very good or 
excellent health.^^ 

Yet where impacts were found (in four of the six programs) they were consistently unfa- 
vorable.^^ Two programs (Grand Rapids LFA and Riverside HCD) decreased the proportion of 
children reported as being in very good or excellent health, reducing these levels by about 7 per- 



group (and/or more likely to move him or her into a lower-level group) if the child tends to be disruptive to other 
students in the class. 

^*The increase in absenteeism in the Atlanta LFA program occurred especially among the least disadvantaged 
subgroup, a finding that was part of a broader pattern of unfavorable impacts for this subgroup. 

^^Adams and Marano, 1995. It may be surprising that mothers in the COS sample reported higher health ratings 
for their children aged 8 to 10 than were reported for both all children aged 5 to 17 and those from low-income fami- 
lies in a national sample. However, mothers were exempt from participation in JOBS welfare-to-work activities if 
they were needed at home to care for an ill or incapacitated family member, including a child. Consequently, the 
COS sample of children is relatively healthy, whereas national sanples of children would include some severely and 
chronically ill children. 

^^The pattern of unfavorable health impacts, including some new impacts in subgroups that were masked in the 
aggregate, was found to be largely concentrated among boys. 
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National Evaluation of Welfare-to-Work Strategies 
Table 12.4 

Impacts on Health and Safety 



Site and Program 


Sanq>le 

Size 


Program 

Group 


Control 

Group 


Difference ] 

(Impact) 


Effect 

Size 




General health ratine franee of 1 to 5) (mother report) 


Atlanta Labor Force Attachment 


568 


4.2 


4.3 


0.0 


-0.08 


Atlanta Human Capital Development 


641 


4.3 


4.3 


0.1 


0.09 


Grand Rapids Labor Force Attachment 


386 


4.3 


4.3 


-0.1 


-0.10 


Grand Rapids Human Capital Development 


365 


4.3 


4.3 


0.0 


0.07 


Riverside Labor Force Attachment 


473 


4.2 


4.3 


-0.1 


-0.11 


Lacked high school diploma or basic skills 


299 


4.2 


4.2 


0.0 


0.07 


Riverside Human Capital Development 


395 


4.1 


4.2 


-0.1 


-0.16 




In very eood or excellent health (%) (mother report) 




Atlanta Labor Force Attachment 


568 


81.8 


85.2 


-3.4 


-0.12 


Atlanta Human Capital Development 


641 


83.6 


85.8 


-2.1 


-0.07 


Grand Rapids Labor Force Attachment 


386 


83.4 


90.4 


-7.0 ** 


-0.26 


Grand Rapids Human Capital Development 


365 


87.0 


90.1 


-3.2 


-0.12 


Riverside Labor Force Attachment 


473 


85.7 


87.6 


-2.0 


-0.07 


Lacked high school diploma or basic skills 


299 


85.1 


82.7 


2.5 


0.08 


Riverside Human Capital Development 


395 


75.6 


83.0 


-7.4 * 


-0.24 




Has a phvsicaK 


emotional. 


. or mental condition 






that reauires frequent medical attention, use of medication. 






or special equipment (%) (mother report) 




Atlanta Labor Force Attachment 


600 


6.6 


2.7 


3 9** 


0.30 


Atlanta Human Capital Development 


678 


6.9 


2.5 


4 4 ♦♦♦ 


0.34 


Grand Rapids Labor Force Attachment 


428 


13.6 


12.8 


0.8 


0.02 


Grand Rapids Human Capital Development 


410 


13.4 


13.2 


0.2 


0.01 


Riverside Labor Force Attachment 


53.3 


6.7 


7.3 


-0.6 


-0.02 


Lacked high school diploma or basic skills 


338 


6.3 


6.5 


-0.2 


-0.01 


Riverside Human Capital Development 


438 


6.5 


6.8 


-0.4 


-0.02 
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327 



- 282 - 



Table 12.4 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Effect 

Size 




Has a phvsicaL emotionak or mental condition 








that impedes on mother's ability to so to 








work or school (%) (mother report) 




Atlanta Labor Force Attachment 


600 


4.5 


1.2 


3 3 ♦♦ 


0.34 


Atlanta Human Capital Development 


678 


3.1 


1.3 


1.8 


0.18 


Grand Rapids Labor Force Attachment 


428 


6.4 


8.1 


-1.7 


-0.07 


Grand Rapids Human Capital Development 


410 


6.3 


8.3 


-2.0 


-0.08 


Riverside Labor Force Attachment 


533 


4.0 


6.2 


-2.2 


-0.10 


Lacked high school diploma or basic skills 


338 


3.1 


5.0 


-1.9 


-0.10 


Riverside Human Capital Development 


438 


5.3 


5.2 


0.1 


0.01 




Had an 


accident or iniurv reauirins emersencv medical 




attention, since the last interview (Vo) (mother report) 


Atlanta Labor Force Attachment 


600 


15.7 


16.4 


-0.7 


-0.02 


Atlanta Human Capital Development 


678 


15.3 


15.9 


-0.6 


-0.02 


Grand Rapids Labor Force Attachment 


428 


26.3 


23.6 


2.7 


0.07 


Grand Rapids Human Capital Development 


409 


22.4 


24.2 


-1.8 


-0.04 


Riverside Labor Force Attachment 


532 


23.2 


25.9 


-2.7 


-0.07 


Lacked high school diploma or basic skills 


337 


20.9 


21.2 


-0.3 


-0.01 


Riverside Human Capital Development 


435 


20.4 


21.6 


-1.3 


-0.04 



SOURCE: Child Trends calculations from the Five-Year Child Outcomes Study survey (mother reports). 
NOTES: See Appendix A. 2. 
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centage points. It is worth noting, however, that these unfavorable impacts resulted from program 
mothers being more likely than control group mothers to rate the focal child as being in good — 
not fair or poor — health. In addition, both Atlanta programs unfavorably affected two measures 
of health, increasing the proportion of children whose mothers reported that they had a physical, 
emotional, or mental condition that required frequent medical attention or the use of medication 
or special equipment and increasing the proportion with such a condition that impeded mothers’ 
ability to go to work or school (although the difference for the HCD program on the latter meas- 
ure was just beyond the cutoff for statistical significance). The impacts on conditions requiring 
frequent medical attention or the use of medication or special equipment were each about 4 per- 
centage points. The impacts on conditions impeding the mothers’ work or schooling were gener- 
ally smaller, about 3 percentage points in the LFA program and about 2 percentage points in the 
HCD program. These health impacts may shed light on the unfavorable impacts on absenteeism 
and tardiness that were found in the Atlanta programs and in the Riverside HCD program. It is 
possible that the decreases in health status caused by these programs led to an increased likeli- 
hood of a child’s being absent or tardy. 

D. Other Outcome 

There were no aggregate impacts on the likelihood that focal children lived away from 
their mothers because they could not care for them. Between 3 percent (in Atlanta) and 9 per- 
cent (in Riverside) of program group mothers reported separation from focal children for this 
reason, levels that did not differ significantly from those reported by the control groups in any 
site. (See Table 12.5.) 

IV. “Mapping” Child and Adult Impacts Found in the Child Outcome 
Study Sample 

There were relatively few impacts on young children’s developmental outcomes at the 
five-year point; possible reasons why more numerous impacts on children were not found are ad- 
dressed in the next section. Nevertheless, there were more statistically significant impacts than 
would be expected by chance, indicating that welfare-to-work programs can (though modestly) 
alter children’s developmental outcomes — though in different ways across programs, sites, and 
developmental domains. 

This section addresses the question: What may have led to the program impacts on chil- 
dren that were found at the five-year point? The sophisticated statistical analyses necessary to 
answer this question definitively are beyond the scope of the chapter. However, any such “path- 
ways” analyses would need to begin with an examination of impacts on outcomes hypothesized 
to serve as pathways through which children were affected — for example, employment, income, 
and child care and map these impacts onto the impacts found for children. If, for example, 
the programs producing predominantly favorable impacts on young children were the same pro- 
grams that also increased mothers’ employment, this may suggest that increasing employment is 
one way to improve child outcomes. If, in addition, the programs that did not produce favorable 
impacts on children also did not increase employment, then increasing employment would appear 
to be a necessary condition for improving child outcomes. To address this issue, the following 
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National Evaluation of Welfare-to-Work Strategies 
Table 12.5 



Impacts on an Outcome Related to Living Arrangements 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Effect 

Size 




Lived apart from mother, since last interview, 
because she could not care for child (mother report)^ 


Atlanta Labor Force Attachment 


600 


2.5 


4.6 


-2.1 


-0.12 


Atlanta Human Capital Development 


678 


3.1 


4.9 


-1.9 


-0.11 


Grand Rapids Labor Force Attachment 


428 


6.0 


5.9 


0.1 


0.00 


Grand Rapids Human Capital Development 


410 


4.6 


7.0 


-2.4 


-0.11 


Riverside Labor Force Attachment 


533 


8.6 


6.4 


2.2 


0.10 


Lacked high school diploma or basic skills 


338 


6.1 


6.2 


-0.1 


-0.01 


Riverside Human Capital Development 


438 


5.0 


6.8 


-1.7 


-0.09 



SOURCE: Child Trends calculations from the Five-Year Child Outcomes Study survey (mother reports). 
NOTES: See Appendix A.2. 
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section describes the patterns of key adult impacts that may explain the larger pattern of child 
impacts across programs.^'’ 

A. Educational Attainment 

There is not a one-to-one relationship between impacts on mothers’ educational attain- 
ment and impacts on young children across programs. Given the literature on the importance of 
mothers education for children’s developmental outcomes, one would expect increases in moth- 
ers’ education to bode well for children.^^ On the one hand, this appears to be the case; Both the 
Atlanta and the Riverside HCD programs increased mothers’ receipt of an education credential, 
and impacts of these programs on young children were favorable, whereas the Grand Rapids LFA 
program decreased receipt of such a credential, and impacts of this program on young children 
were unfavorable. On the other hand, increases in mothers’ education do not appear necessary for 
improving child outcomes. The Atlanta LFA program did not increase mothers’ educational at- 
tainment, nor did the Riverside LFA program for the subgroup of mothers lacking basic skills at 
baseline, yet these programs also improved child outcomes. Similarly, the Grand Rapids HCD 
program did not alter mothers’ educational attainment but, like the LFA program in this site, it 
led to uniformly unfavorable impacts on children. Thus, impacts on mothers’ education may, in 
part, underlie impacts on children in some programs, but not in other programs. 

B. Employment and Earnines 

The pattern of impacts on mothers’ employment and earnings also does not appear to ex- 
plain differences in child impacts across programs. For instance, while both Riverside programs 
produced relatively large gains in both earnings and employment (with the LFA program having 
these favorable effects both for the full sample and for the subgroup of mothers lacking a high 
school diploma at baseline), the pattern of child impacts varied according to mothers’ education 
level, with few, but favorable, child impacts for children of mothers without a diploma and 
somewhat more numerous and unfavorable impacts for children of mothers in the full sample. 
Further, there were increases of a similar magnitude in earnings in both Atlanta programs and the 
Grand Rapids LFA program, yet the Atlanta programs generally improved child outcomes 
whereas the Grand Rapids programs worsened them. Hence, it is not likely that changes in em- 
ployment and earnings can account for the different pattern of child impacts across programs. 

C. Cumulative Income and Poverty Status 

Some studies have shown that greater income is related to better child outcomes, though 
other studies have found that increased income is most beneficial when it lifts families out of 
poverty, since families with income below the, poverty line are not likely to have sufficient re- 
sources to meet their basic needs.^^ However, there is no apparent relationship in the COS sam- 



^^The adult economic impacts discussed in this section pertain to adults in the COS. See Appendix I for detailed 
tables of these impacts. The smaller sample sizes of the COS at times led differences that were of a similar magni- 
tude to significant impacts in the larger evaluation samples to be nonsignificant in the COS san^le. Hence, this dis- 
cussion reports on differences in the COS that were either statistically significant or of a similar or greater size than a 
significant difference in the larger sample of adults within a given program. 

^See Duncan and Magnuson, 200 1 . 

^^Duncan and Brooks-Gunn, 1997. 
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pie between impacts on total combined income across years 1 through 5, or on family poverty 
status in year 5, and impacts on young children. First of all, there were no impacts on total com- 
bined income in the COS sample, so income cannot be the driving force behind either the favor- 
able or the unfavorable impacts on children. With respect to impacts on poverty status at the five- 
year point, the only program to significantly influence poverty levels, the Riverside HCD pro- 
gram, reduced poverty and improved functioning among focal children. However, the other pro- 
grams had no significant influences on poverty, providing little support for the hypothesis that 
changes in poverty status in year 5 of the follow-up are the driving force in impacts on children. 
It is important to note, however, that this measure of poverty status does not capture program im- 
pacts on cumulative poverty over the five-year period. Yet previous research suggests that chil- 
dren’s cumulative poverty experiences are more important for their developmental outcomes than 
is their poverty status in any given year.^^ Hence, although these findings suggest that poverty at 
the time of the follow-up is not likely to account for the pattern of impacts on children, we can 
draw few inferences about the role of cumulative poverty in accounting for these impacts. 

D. Child Care 

The pattern of impacts in the COS sample on measures of employment-related child care 
does not appear to explain differences in child impacts across programs. There were increases in the 
use of child care after leaving welfare (because of earnings) in both programs in Atlanta and River- 
side, and in the LFA program in Grand Rapids; however, these programs did not have uniformly 
favorable or unfavorable impacts on children. Likewise, when impacts on the use of transitional 
child care benefits emerged in the COS sample, the programs increased the use of this benefit, yet 
impacts on young children within these programs were, again, both favorable and unfavorable. 

Impacts on young children’s supervision and child care during nonschool hours occurred 
almost exclusively in the Riverside site and, therefore, cannot explain the pattern of child impacts 
found across the programs. 

V. Discussion and Conclusions 

The major goal of the Child Outcomes Study was to determine whether mandated partici- 
pation in a welfare-to-work program could have implications for enrollees’ children — particu- 
larly preschool-age children whose mothers were newly required by the Family Support Act of 
1988 to engage in work preparation activities as a condition of receiving welfare. Some policy- 
makers feared that requiring mothers of preschool-age children to secure employment would 
harm their children; others argued that the economic benefits of employment that would accrue to 
these families would benefit children. Findings from the COS indicate that there were relatively 
few long-term impacts of the JOBS programs evaluated in NEWWS on children’s developmental 
outcomes. The lack of pervasive impacts on children is perhaps not surprising, given that the 
JOBS program was not aimed at children. It may be that impacts on outcomes important to chil- 
dren, such as mothers’ employment, family income, and/or the children’s immediate environ- 
ments (home, school, child care), were too few, occurred for too brief a period, or were of an in- 
sufficient magnitude to lead to consistent, enduring impacts on young children. It is also possible 



^’Duncan and Brooks-Gunn, 1997. 



O 



-287- 



332 



that impacts at the five-year point are understated in Atlanta and Grand Rapids because some 
control group members in these sites became eligible for welfare-to-work program services in 
years 4 or 5 of the follow-up period. 

However, the number of impacts found exceeds chance levels, and there was a discernible 
pattern of impacts on young children.^^ At the five-year follow-up, all six JOBS programs exam- 
ined affected children s social skills and behavior. Both Atlanta programs increased positive behav- 
iors, and decreased problem behaviors; both Grand Rapids programs decreased positive behaviors, 
and the HCD program increased problem behaviors; and the Riverside LFA program increased 
problem behaviors in the full sample.^^’^*^ For young children whose mothers lacked a high school 
diploma or basic skills at study entry, both programs in Riverside increased positive behaviors. 
Changes in problem behaviors may be especially important; research has shown that a history of 
antisocial behavior in childhood which is partially measured by the problem behaviors examined 
in this study — is the strongest risk factor for chronic delinquency among adolescents.^' 

Impacts on young children’s academic achievement were also relatively rare, but findings 
relating to behavioral adjustment to school (school engagement, disciplinary problems) were in 
accord with the patterns described above, with uniformly favorable impacts in Atlanta and uni- 
formly unfavorable impacts in Grand Rapids. Impacts relating to how children were performing 
academically in school were scarce; however, when impacts were found on these outcomes and 
others, they were at the lower end of the effect size range for early intervention programs aimed 
directly at children. These findings suggest that welfare-to-work programs can affect children’s 
performance in school, albeit not as frequently nor as systematically as they appear to affect chil- 
dren’s behavior. 

Finally, there were also few impacts in the domain of health and safety; however, they 
were all unfavorable, occurred in four of the six programs,^^ and tended to occur in programs that 
also increased children’s school absenteeism and/or tardiness. In many cases, these unfavorable 



28 

Impacts on young children two years after enrollment (when focal children were aged approximately 5 to 7) 
tended to vary by developmental domain, with only favorable aggregate impacts in the cognitive domain and only 
unfavorable impacts in the health and safety domain. Impacts on young children’s behavior and socioemotional ftmc- 
tioning were both favorable and unfavorable. See McGroder et al., 2000; and Zaslow, McGroder, and Moore, 2000. 

These patterns of impacts for young children do not match findings discussed in Chapter 11, as the client sur- 
vey sample respondents did not provide any information on social skills and behavior for all of their children. Only 
mothers in the COS provided this information for the focal children, as presented in this chapter, and it is precisely 
this domain of child outcomes most consistently affected by these programs. 

As stated previously, the favorable impacts of the Atlanta programs on young children’s positive and problem 
behaviors were especially pronounced for girls. At the same time, the unfavorable impacts of the Riverside LFA 
program on young children’s problem behaviors were also especially pronounced for girls. 

^'Farrington, 1987, as cited in Yoshikawa, 1995. 

Interestingly, as reported in Chapter 11, favorable impacts were found for preschool-age children on whether 
any child had a condition that impeded on the mother’s ability to go to work or school in the Riverside LFA program 
(for both the full sample and for those in need). Approximately two-thirds of the children in this age group were in- 
cluded in the COS sample presented in this chapter. As shown in Table 12.4, though, there were no favorable health 
impacts on this measure for the Riverside LFA program in the COS sample. However, the difference in the means 
between the program and control groups did go in the same direction. Perhaps the larger size of the client survey 
sample (and, thus, smaller standard errors) accounts for the difference in the presence of inqiacts. 
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health impacts were being driven by the unfavorable impacts for boys, with girls’ health left 
largely unaffected by these programs. 

A second goal of the COS was to determine whether program impacts on young children 
of enrollees differed according to the welfare-to-work strategy employed — specifically, a Labor 
Force Attachment or a Human Capital Development strategy. Some believed that increases in 
mothers’ educational attainment resulting from participation in an HCD program would bode 
particularly well for children — particularly for their school success — even if such participation 
did not lead immediately or ultimately to increased employment and earnings. Others argued that 
the quicker that mothers secured employment (through the LFA approach), the quicker that fi- 
nancial benefits would accrue to children. However, contrary to these initial hypotheses, the wel- 
fare-to-work strategy employed did not consistently produce different impacts on children at the 
five-year follow-up. In direct comparisons of LFA and HCD programs, neither approach 
emerged as uniformly better or worse for children. 

Did five-year impacts on children vary according to initial levels of family risk? The 
Family Support Act sought explicitly to reduce long-term welfare dependency by providing the 
services necessary to move long-term recipients into jobs. Many argued that the opportunities 
that JOBS provided would be more beneficial for the most disadvantaged participants (five-year 
impact findings for adults discussed in Chapter 7 support this hypothesis), thereby improving 
outcomes for children. Others feared that higher-risk participants might not be able to mobilize to 
meet JOBS requirements and would face sanctions, and they and their children would subse- 
quently suffer. Contrary to these expectations, findings from the five-year follow-up do not show 
a discernible pattern of favorable or unfavorable impacts — either across programs or within a 
single program — for children of the “most disadvantaged” mothers. Thus, children aged 8 to 
10 in these higher-risk faihilies do not appear to have been consistently helped or hurt by their 
mothers’ enrollment in the LFA and HCD programs. Rather — and somewhat unexpectedly — 
there emerged a concentration of unfavorable impacts on children of the least disadvantaged 
mothers in the Riverside LFA program.^^ Interestingly, the Riverside LFA program unfavorably 
affected the same problem behavior and discipline outcomes for girls, which begs the question of 
whether these unfavorable impacts were concentrated among the least disadvantaged girls 

In the Riverside LFA program, the unfavorable impacts on the academic performance, 
school engagement, and problem behaviors of focal children with the least disadvantaged moth- 
ers found at the five-year point are consistent with those found at the two-year point: increased 



^^Similarly, inqiacts on young children at the two-year follow-up did not vary consistently by welfare-to-work 
Strategy enq)loyed. See Hamilton, 2000; McGroder et al., 2000; and Zaslow, McGroder, and Moore, 2000. 

^'‘The most disadvantaged mothers did not have a high school diploma as of random assignment, did not work in 
the year prior to random assignment, and had been on welfare for at least two years as of random assignment. The 
least disadvantaged mothers had none of these barriers, and the moderately disadvantaged mothers had one or two of 
these barriers. See Chapter 7. . 

Although sanq)les sizes for the least disadvantaged subgroup in these programs were relatively small, impacts 
for this subgroup are being reported because they were relatively numerous and were consistent in direction (they 
were mainly unfavorable). A similar pattern of unfavorable impacts was found for children in lower-risk families 
(defined by greater employment and low levels of welfare receipt) assigned to Florida’s Family Transition Program. 
See Bloom et al., 2000a. 

^^Small san^le sizes precluded testing this hypothesis empirically. 



academic problems, emotional problems, and suspensions or expulsions of children in lower-risk 
families. Other than these impacts, the JOBS programs examined in COS did not appear to have 
a similar pattern of short- and long-term impacts on children from families varying in initial risk. 
The only other pattern of findings to emerge by family risk level at the two-year follow-up^^’^* 
a pattern of favorable (though relatively small) impacts on children in higher-risk families in 
all HCD programs and in the Atlanta LFA program — did not emerge at the five-year point for 
children with the most disadvantaged mothers. 

What led to the program impacts on children at the five-year point? It is worthwhile to note 
what did not lead to these impacts. Recent reviews of experimental findings for children indicate 
that, when welfare-to-work programs increase both maternal employment and family income, chil- 
dren often benefit.^® However, because none of the six JOBS programs examined here affected the 
income of COS sample families, income cannot be a pathway through which children were af- 
fected. Also, use of child care does not appear to underlie program impacts on children. 

There is some evidence from a nonstatistical “mapping” of adult and child impacts at the 
five-year point that maternal education may have been an important pathway through which 
some children were affected by their mothers’ enrollment in an LFA or HCD program. Three of 
these programs affected both maternal education and child outcomes in the same direction; in the 
other three programs, though maternal education was not affected, children were. Thus, changes 
in mothers’ education may not necessarily lead to changes in child outcomes, but when mothers’ 
education is affected, it may have implications for their children. This finding is consistent with 
results showing that increased participation in educational activities by mothers in these three 
COS sites predicted greater academic school readiness in their children.'*® 

In addition to examining the pattern of impacts across programs, it is important to con- 
sider impacts specific to a particular program for identifying possible pathways through which 
children were affected by that program. For example, the Riverside LFA program reduced time 
spent in child care activities after school and decreased the use of formal child care as a regular 
arrangement. Given some emerging evidence on the potential benefits of more formal child care 
arrangements on children,'*' the reduction in the use of formal care may be related to the pattern 
of unfavorable impacts of this program on these young children. In addition, research has shown 
a negative association between family residential moves and children’s behavior.'*^ This may help 
to explain the pattern of unfavorable impacts in Grand Rapids, given that both the LFA and the 
HCD programs in this site increased the likelihood that families had moved since study entry and 
also had unfavorable impacts on social skills and behavior among children aged 8 to 10. In short. 



^’See McGroder et al., 2000. 

These comparisons rely on different child outcomes measured at the two-year and five-year follow-ups (with 
no child or teacher reports in the two-year follow-up) and on different (though similar) characterizations of sub- 
groups. 

^’Zaslow et al., forthcoming. 

““Magnuson and McGroder, 2001. 

“‘NICHD Early Child Care Research Network, 2000; see also U.S. Department of Education, 1995. As dis- 
cussed in Chapter 10, the NICHD findings also suggest that extensive participation in formal child care arrangements 
may have the unfavorable result of increasing children’s externalizing behavior problems. 

“^Zaslow and Eldred, 1998. 
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because program impacts on children represent the net effect of all impacts on outcomes impor- 
tant to their development and well-being, and because impacts on these “intervening” outcomes 
may differ in different programs, future research needs to examine pathways for each program 
separately and needs to consider, simultaneously, all the intervening outcomes that were affected 
by each program."*^ 

What is clear is that the sites in which these six JOBS programs were implemented varied 

— geographically, economically, demographically (including the racial/ethnic composition of the 
caseloads), and in the policies, practices, and ethos of the local welfare offices/'* Differences in 
program implementation may also help to explain the pattern of impacts. For example, though 
the Atlanta site successfully implemented distinct LFA and HCD models (in terms of the content 
and sequence of services), a large proportion of both HCD and LFA case managers in this site 
believed that an education-oriented welfare-to-work strategy was the best means of leaving wel- 
fare.'*^’'*® In addition, case managers in the Atlanta site viewed their roles as very customer- 
oriented; they actively sought out necessary support services for their clients and were less strict 
about monitoring compliance. It may be that the policies, practices, and ethos of the Atlanta wel- 
fare office “fit” well with the needs of its female clients to balance both work and family respon- 
sibilities and that this context helps to explain the beneficial results for children in both the LFA 
and the HCD programs. 

Although a national program, the JOBS program was implemented locally — in sites that 
differed in economic conditions, the population served, and in the ethos and practices of the wel- 
fare offices. These conditions, though not included in the statistical analyses reported on here, 
shape the way the JOBS programs were implemented in each site and, thus, can shape the im- 
pacts that these programs have on targeted and nontargeted adult outcomes and on child out- 
comes in as yet unknown ways. Future research on the effectiveness of welfare-to-work programs 

— for both generations — may need to focus increasingly on identifying what works for whom, 
and under what circumstances. Such research would be informative to state and local policymak- 
ers and program planners who increasingly bear the responsibility, in this age of devolution, to 
design and effectively target welfare-to-work programs. 



■‘bindings from the two-year COS suggest that young children may be affected through multiple pathways, that 
pathways can vary according to the child outcome considered, and that different programs may activate different 
pathways in affecting children. See McGroder et al., 2000. 

"^"^Hamilton and Brock, 1994. 

'^^Hamilton et al., 1997. 

"^^This may be one reason that only the LFA program in Atlanta improved mothers’ educational attainment by 
the two-year follow-up. See McGroder et al., 2000; Zaslow et al., 2000. 
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Chapter 13 

Costs and Benefits 



Earlier chapters described the implementation of the NEWWS programs and the effects 
on sample members and their families. In sum, NEWWS programs increased participation in 
employment-related activities and produced some positive effects for program group members, 
including higher earnings and less reliance on welfare. Overall, the NEWWS programs had 
little impact positive or negative — on the well-being of the children of program group 
members. This chapter presents an analysis of the costs of providing the array of services that 
produced these effects and uses the results of the cost analysis to examine the net financial 
benefits and costs of the NEWWS programs from two perspectives: program group members 
and government budgets. The benefit-cost analysis includes key financial effects discussed in 
earlier chapters, such as effects on earnings, cash assistance payments, and Food Stamp pay- 
ments, and expands the scope to consider effects such as fringe benefits from employment, 
taxes, and Medicaid coverage. 

This chapter extends the analysis of impacts on use of program services (Chapter 3) and 
impacts on earnings, welfare and Food Stamps, and combined income (Chapters 4-6). It exam- 
ines the costs and benefits of 10 of the 1 1 NEWWS programs, those for which five years of fol- 
low-up data were available from administrative records. (Owing to data limitations five-year 
costs and benefits were not estimated for Oklahoma City.) 

The chapter presents details on the analysis of the costs of running the NEWWS pro- 
grams, discusses the financial benefits of NEWWS, and compares the benefits and costs from the 
two perspectives mentioned above. 

I. Key Findings 

• Five-year gross costs for employment-focused programs were similar to 
those for education-focused programs in the 10 NEWWS programs exam- 
ined. Although the Grand Rapids LFA and Portland programs were employ- 
ment-focused, they produced participation rates in post-secondary and voca- 
tional training that were similar to or greater than those found in t)ie educa- 
tion-focused programs. The average cost was $7,749 for the employment- 
focused programs and $7,499 for the education-focused programs. 

The costs presented in this chapter consist of all costs associated with providing employ- 
ment services and related support services to sample members. The gross cost per program group 
member consists of costs paid by the welfare department and non-welfare agencies while sample 
members were enrolled in NEWWS programs, as well as for employment and support services 
after they exited the programs and left the welfare rolls. 

• The net costs of the NEWWS programs, over and above the cost of ser- 
vices used by control group members, were higher for education-focused 
programs ($3,972) than for employment-focused ($3,037). 
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The net cost per program group member is the gross cost per program group member mi- 
nus what would have been spent in the absence of the NEWWS program — the gross cost per 
control group member. As discussed in Chapter 3, a sizable number of controls participated on 
their own in community-provided employment-related activities, usually vocational training or 
post-secondary education. 

• Gross and net costs were higher for the NEWWS programs than for 
other programs studied by MDRC. 

This is not surprising, given the high rates of participation in higher-cost education activi- 
ties, such as post-secondary education and vocational training. The average cost of the NEWWS 
programs was comparable to other high-cost programs: the Alameda and Los Angeles Greater 
Avenues for Independence (GAIN) programs operated in the late 1980s and early 1990s. 

• As was the case at the two-year follow-up point, the net costs of the HCD 
programs continued to be higher than those of the LFA programs — by 
about 40 to 90 percent. 

This comparison is based on the three sites where head-to-head LFA-HCD tests were 
conducted: Atlanta, Grand Rapids, and Riverside. Because the Riverside HCD sample includes 
only those who did not have a high school diploma or GED when they entered the evaluation, the 
LFA-HCD comparison in this site is limited to this subgroup. 

• Generally, programs spent more on serving those who entered the evalua- 
tion with a high school diploma or GED than those who entered without 
these credentials. 

In all programs, gross costs per program group member were higher for the subgroup that 
entered the evaluation with a high school diploma or GED. As expected, they were more likely to 
participate in post-secondary education or vocational training activities, whereas those lacking 
these credentials were more likely to participate in basic education activities, which typically cost 
less. The differences in net costs between the subgroups were less pronounced, but still higher for 
those with a high school diploma or GED. 

From the perspective of the welfare recipients subject to the programs, the benefit-cost 
findings show that: 

• The three programs that produced the smallest welfare reductions (under 
$1,000 for both Atlanta programs and the Detroit program) resulted in 
net gains to sample members, albeit very small ones; sample members ef- 
fectively broke even in these programs. All programs with larger five-year 
welfare reductions (ranging from $1,148 in the Columbus Traditional program 
to $2,868 in the Riverside LFA program) produced net five-year losses for 
sample members. 

• A similar pattern was found for the subgroup of sample members who 
entered the evaluation with a high school diploma or GED, although the 
Columbus programs, with similarly small welfare reductions, produced 
net losses. Those who entered without these education credentials generally 
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suffered net losses. Detroit and Portland were the exceptions: Those without a 
high school diploma or GED experienced a moderate gain in Portland, and 
those in Detroit effectively broke even. 

From the perspective of government budgets, the findings show that: 

• For the full samples, only three programs, all employment-focused, pro- 
duced gains to government budgets (Grand Rapids LFA, Riverside LFA, 
and Portland). All programs that produced the smallest welfare reductions, 
making it difficult for government budgets to break even, resulted in losses. 

Grand Rapids HCD and Columbus Integrated, with midrange welfare savings, 
effectively broke even. 

• In terms of the return to budget per net dollar invested, on average the 
results from the government perspective are more positive than those 
found in the GAIN evaluation. The NEWWS programs averaged a return of 
$1.29; the GAIN programs averaged $0.76 per dollar invested. Moreover, the 
two programs with the highest returns, Portland ($2.83) and Grand Rapids 
LFA ($2.46), compare favorably with the Riverside GAIN program and the 
Saturation Work Initiative Model (SWIM) program, which returned $2.84 and 
$2.34, respectively. 

• With the exception of the two programs in Atlanta, government budgets 
recouped their investments in sample members who entered the evalua- 
tion without a high school diploma or GED. Among sample members who 
entered the evaluation with a high school diploma or GED, three of the four 
employment-focused programs produced gains to government budgets (At- 
lanta LFA broke even) and all of the education-focused programs produced 
losses. 

II. Issues in the Cost Analysis 

The primary purpose of the cost analysis is to estimate the cost of providing NEWWS 
services, over and above the cost that would have been incurred in the absence of the program — 
that is, to estimate the average net cost per program group member. The net cost is the difference 
between the gross cost per program group member and the gross cost per control group mem- 
ber, where the gross costs reflect the cost of all services that sample members used in the 
NEWWS programs and of the education and training services that they used outside the pro- 
grams, when they were no longer receiving welfare benefits. In other words, the cost for the con- 
trol group is the benchmark used to determine the additional costs incurred as a result of the 
NEWWS programs. 

This report updates cost figures based on two years of follow-up data presented in earlier 
reports.' (As noted above, data limitations did not allow inclusion of Oklahoma City in the five- 
year analysis.) Table 13.1 shows that, at two years, education-focused programs were more costly 



'See Hamilton et al., 1997; Scrivener et al., 1998; Farrell, 2000; Storto et al., 2000; and Scrivener and Walter 
2001. 



National Evaluation of Welfare-to-Work Strategies 
Table 13.1 



Estimated Gross and Net Costs Within a Two-Year Follow-Up Period, 
by Program and Service Component (in 1999 dollars) 





Total Gross Cost 


Total Gross Cost 


Total Net Cost 




per Program Group 


per Control Group 


per Program Group 


Program and Component 


Member ($) 


Member ($) 


Member ($) 


Atlanta Labor Force Attachment 








Orientation and appraisal 


72 


n/a 


72 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


816 


8 


807 


Basic education 


507 


58 


449 


Post-secondary education 


289 


315 


-26 


Vocational training 


751 


443 


309 


Work experience 


174 


20 


152 


Subtotal (operating) 


2,609 


843 


1,765 


Child care 


883 


291 


591 


Other support services 


193 


17 


176 


Total 


3,685 


1,152 


2,533 


Atlanta Human Capital Development 








Orientation and appraisal 


72 


n/a 


72 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


194 


8 


185 


Basic education 


1,263 


58 


1,205 


Post-secondary education 


300 


315 


-14 


Vocational training 


1,792 


443 


1,350 


Work experience 


126 


20 


105 


Subtotal (operating) 


3,746 


843 


2,903 


Child care 


806 


291 


515 


Other support services 


413 


17 


397 


Total 


4,966 


1,152 


3,814 


Grand Rapids Labor Force Attachment 






Orientation and appraisal 


18 


n/a 


18 


Formal assessment 


9 


n/a 


9 


Job search 


823 


78 


745 


Basic education 


819 


793 


27 


Post-secondary education 


1,944 


1,715 


229 


Vocational training 


730 


841 


-111 


Work experience 


122 


12 


no 


Subtotal (operating) 


4,465 


3,438 


1,026 


Child care 


408 


230 


177 


Other support services 


30 


n/a 


30 


Total 


4,903 


3,669 


1,233 



(continued) 
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Table 13.1 (continued) 



Total Gross Cost 
per Program Group 

Program and Component Member ($) 


Total Gross Cost 
per Control Group 
Member ($) 


Total Net Cost 
per Program Group 
Member ($) 


Grand Rapids Human Capital Development 








Orientation and appraisal 


18 


n/a 


18 


Formal assessment 


312 


n/a 


312 


Job search 


289 


78 


210 


Basic education 


2,064 


793 


1,272 


Post-secondary education 


2,123 


1,715 


408 


Vocational training 


1,205 


841 


364 


Work experience 


213 


12 


201 


Subtotal (operating) 


6,224 


3,438 


2,786 


Child care 


603 


230 


372 


Other support services 


37 


n/a 


37 


Total 


6,865 


3,669 


3,195 


Riverside Labor Force Attachment 








Orientation and appraisal 


111 


n/a 


111 


Formal assessment 


6 


n/a 


6 


Job search 


877 


45 


832 


Basic education 


172 


106 


67 


Post-secondary education 


707 


421 


286 


Vocational training 


237 


243 


-6 


Work experience 


56 


65 


-10 


Subtotal (operating) 


2,164 


878 


1,286 


Child care 


98 


33 


66 


Other support services 


54 


n/a 


54 


Total 


2,316 


911 


1,405 


Riverside Human Capital Development 








(without a high school diploma or GED) 








Orientation and appraisal 


107 


n/a 


107 


Formal assessment 


13 


n/a 


13 


Job search 


729 


43 


685 


Basic education 


2,340 


181 


2,158 


Post-secondary education 


186 


150 


36 


Vocational training 


236 


258 


-22 


Work experience 


63 


30 


33 


Subtotal (operating) 


3,674 


662 


3,012 


Child care 


183 


16 


166 


Other support services 


82 


n/a 


82 


Total 


3,938 


678 


3,260 



(continued) 
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Table 13.1 (continued) 



Program and Component 


Total Gross Cost 
per Program Group 
Member ($) 


Total Gross Cost 
per Control Group 
Member ($) 


Total Net Cost 
per Program Group 
Member ($) 


Columbus Integrated 








Orientation and appraisal 


17 


n/a 


17 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


79 


7 


72 


Basic education 


516 


89 


427 


Post-secondary education 


1,163 


270 


892 


Vocational training 


693 


226 


467 


Work experience 


82 


7 


76 


Subtotal (operating) 


2,550 


599 


1,952 


Child care 


569 


346 


221 


Other support services 


221 


11 


211 


Total 


3,340 


956 


2,384 


Columbus Traditional 








Orientation and appraisal 


9 


n/a 


9 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


100 


7 


93 


Basic education 


632 


89 


543 


Post-secondary education 


1,046 


270 


775 


Vocational training 


313 


226 


87 


Work experience 


63 


7 


57 


Subtotal (operating) 


2,163 


599 


1,564 


Child care 


552 


346 


205 


Other support services 


148 


11 


138 


Total 


2,863 


956 


1,907 


Detroit 








Job search 


265 


47 


218 


Education and training^ 


3,541 


1,890 


1,651 


Work experience 


. 78 


10 


68 


Subtotal (operating) 


3,884 


1,947 


1,938 


Child care 


407 


337 


71 


Other support services 


61 


17 


45 


Total 


4,354 


2,300 


2,053 



(continued) 




-297- 

1 * ' 



342 



Table 13.1 (continued) 



Program and Component 


Total Gross Cost 
per Program Group 
Member ($) 


Total Gross Cost 
per Control Group 
Member ($) 


Total Net Cost 
per Program Group 
Member ($) 


Oklahoma City 


Orientation and appraisal 


n/a 


n/a 


n/a 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


127 


46 


80 


Basic education 


553 


276 


111 


Post-secondary education 


814 


738 


11 


Vocational training 


1,048 


625 


423 


Work experience 


34 


14 


20 


Subtotal (operating) 


2,576 


1,698 


878 


Child care 


593 


506 


87 


Other support services 


87 


n/a 


87 


Total 


3,255 


2,204 


1,051 


Portland 


Orientation and appraisal 


149 


n/a 


149 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


468 


47 


422 


Basic education 


543 


337 


206 


Vocational training and 
post-secondary education 


1,358 


1,159 


199 


Work experience 


247 


37 


210 


Subtotal (operating) 


2,766 


1,580 


1,186 


Child care 


1,582 


629 


953 


Other support services 


79 


6 


73 


Total 


4,427 


2,215 


2,212 



SOURCES: MDRC calculations based on fiscal and participation data from the following: Atlanta - the Fulton 
County Department of Family and Children Services, the Georgia Department of Human Resources, the Georgia 
Department of Technical and Adult Education, the Board of Regents University System of Georgia; Grand 
Rapids - the Michigan Department of Social Services, the Michigan Department of Education Office of 
Extended Learning Services, the Grand Rapids Community College, the Wyoming Community Education 
Center; Riverside - the California Department of Social Services, the California Department of Education, the 
Chancellor’s Office of California Community Colleges; Columbus - the Franklin County Department of Human 
Services, the Ohio Department of Education, the Office of Vocational and Adult Education, the Ohio Board of 
Regents, the National Center for Education Statistics; Detroit - the Michigan Family Independence Agency, the 
Michigan Department of Education, the Michigan Jobs Commission; Portland - the Oregon Department of 
Human Resources, Adult and Family Services Division, the Oregon Office of Community College Services; in 
all sites - information collected on tuition charged at proprietary schools attended by sample members, from 
MDRC-collected JOBS case file data, and the MDRC Two-Year Client Survey. MDRC child care and other 
support service calculations from Fulton County, Kent County, Riverside County, Ohio Department of Human 
Services, Wayne County, Washington County, and Multnomah County (District 2) payment data. Other support 
service data from county records. 
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(continued) 



Table 13.1 (continued) 



NOTES: Rounding may cause slight discrepancies in calculating sums and differences. 

N/a = not applicable. 

“Owing to data limitations, in Detroit it was not possible to separate vocational training, post-secondary 
education, and basic education costs. Thus, the gross costs of all three activities are included in the cost of 
"education and training." Furthermore, in this site orientation and assessment costs are spread across these three 
activities. 
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to operate than employment-focused programs. The average net cost was $1,846 (in 1999 dol- 
lars) for the employment-focused programs and $2,523 for the education-focused programs. 
Similarly, the net costs were 1.5 to 2.6 times greater for the HCD programs than for the LFA 
programs. The earlier reports presented only program costs because a full cost-benefit accounting 
with only two years of data was considered premature, in that the total return on program invest- 
ments would be evident only after several years. 

As described in the earlier reports, costs per sample member are the product of unit costs 
and behavioral variables. The unit cost of an activity is an estimate of the average cost of serving 
one person in a specified activity for a specified unit of time (one month or one hour, for example). 
In general, unit costs were calculated by dividing expenditures for an activity (or service) during a 
steady-state period by the total number of participant-months in that activity during the same pe- 
riod. The number of participant-months was obtained by counting the number of participants in an 
activity in each month of the steady-state period and summing across the months.^ Once the unit 
cost of an activity was determined, it was multiplied by the average number of months that sample 
members spent in the activity, called the behavioral variable, to determine the average cost incurred 
per program group member or control group member during the follow-up period. 

The costs presented here were calculated using the same unit costs calculated at the two- 
year point. However, the behavioral variables used in this kialysis cover the five-year period fol- 
lowing each sample member’s entry into the study. Behavioral variables for years 3 to 5 of fol- 
low-up were estimated using Two-Year and Five-Year Client Survey data, as well as administra- 
tive records data. The Five-Year Client Survey was not administered in Columbus and Detroit. 
Therefore, in Columbus, behavioral variables were estimated using participation trends fi'om the 
two-year and five-year follow-up points in the education-focused sites where longer-term data 
were available. Because Detroit shifted program focus fi-om education to employment mid- 
follow-up, it was considered inappropriate to estimate participation for this program using data 
fi'om the other education-focused programs. Three years of participation data were available fi'om 
the management information system maintained by the Detroit Work First program. Participation 
in years 4 and 5 of the follow-up period was estimated based on patterns of participation and wel- 
fare receipt in the earlier period. 

As discussed in Chapter 2, in several sites service embargoes for control group members 
were lifted during the follow-up period. As a result, in those sites some control group members 
were required to participate in employment-related activities in follow-up year 4 or 5. Welfare 
receipt data were used to determine whether participation should be counted as in-program or 
out-of-program for both program group members and control group members for whom embar- 
goes were lifted. In addition to the cost of providing the activity, in-program participation incurs 
an additional cost for case management. 

As noted above, costs are estimated for the five-year period following sample members’ 
entrance into the study. Later in the chapter, to assess whether the programs in the NEWWS 
Evaluation were cost-effective from the perspective of the government’s budget, this five-year 
net cost is compared with the value of any budgetary savings during the same period (for exam- 



2 

See the two-year reports mentioned in footnote 1 for more detailed information on the calculation of unit costs. 
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pie, from lower welfare or Food Stamp payments) and of any tax revenue increases associated 
with the additional earnings of program group members. ' 

The costs presented here include the costs of program services as well as the costs of em- 
ployment-related services that sample members used outside the programs when they were not re- 
ceiving welfare. The off-welfare costs are important because they represent an additional invest- 
ment of resources that could have differentially affected program and control group members’ fu- 
ture earnings and welfare receipt (effects that are accounted for in the benefit-cost analysis). 

All sample members, not just those who participated in mandatory welfare-to-work 
program services, were included in calculating the net costs because the requirement to par- 
ticipate may have affected some recipients’ behavior: Some people may have chosen to avoid 
the participation mandate by finding a job on their own or by leaving the welfare rolls. In addi- 
tion, sample members who did not participate in welfare-to-work program services may have 
taken part in education and training services on their own, and these costs need to be taken into 
account as well.^ 

Owing to the fact that findings might be expected to differ according to sample members’ 
educational background, all results are presented for the full samples for each program and then 
separately for those with and without a high school diploma or GED at the time they entered the 
evaluation. As noted in earlier chapters, all HCDs in Riverside had no high school diploma or 
GED. Therefore, the Riverside LFA-HCD comparisons include only those without a high school 
diploma or GED. 

III. Major Components of the Cost Analysis 

The costs presented in this chapter consist of all costs associated with providing employ- 
ment services and related support services to sample members. Figure 13.1 illustrates the cost 
components. For each group of sample members, costs were calculated for employment-related 
services that sample members participated in when they were receiving welfare (in-program) and 
services that they participated in when they were not receiving welfare (out-of-program). The 
employment-related services are divided into those that were paid for by the welfare department, 
either directly or indirectly, and those that were paid for by other agencies in the community; this 
information may be useful to administrators and planners who want to understand the nature of 
the government’s investment. 

The remainder of the cost portion of the chapter follows the organization of Figure 13.1, 
beginning with in-program expenditures paid by the welfare departments and ending with the net 
cost per program group member, which is the sum of the program costs less the costs of services 
used by control group members. 



^As noted in Chapter 2, unlike those in other programs in this analysis, Columbus sample members were ran- 
domly assigned to the two programs prior to an orientation. These differences in sample composition are reflected in 
the cost and benefit outcomes. 
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IV. Expenditures for In-Program Employment-Related Services 



Across the programs, costs varied widely for employment-related services for sample 
members when they were receiving welfare — from less than $4,000 in the Riverside LFA and 
Detroit programs to more than $7,000 in the Grand Rapids and Atlanta HCD programs. The 
welfare department paid some of these costs, with the remainder picked up by various other 
agencies in the community. The proportion that was paid for out of welfare department funds 
varied across programs, from 27 percent in the Grand Rapids LFA program to 63 percent in the 
Portland program. This section examines these expenditures in more detail. (See Figure 13.1, 
Boxes 1 and 2.) 

A. Welfare Department Expenditures 

Welfare department costs consisted of program operating costs and the costs of support 
services that sample members received to enable their participation in employment and employ- 
ment-related activities. 

1. Operating costs. The welfare departments paid for the day-to-day operation of the 
NEWWS programs, including expenditures for employment-related case management services, 
overhead, program orientation, and other activities. These expenditures cover services provided 
directly by welfare department staff as well as services provided by other agencies under contract 
to the welfare departments. Welfare department staff directly provided case management (follow- 
ing up on recipients who failed to attend scheduled appointments, providing employability plan- 
ning, and referring and monitoring individuals assigned to activities). In some sites, assessments, 
job search assistance, and job development services were provided by welfare department staff, 
but in others these activities were provided by contracted outside agencies. Costs incurred by the 
welfare department to accommodate NEWWS research requirements and requests were excluded 
from the analysis. 

Table 13.2 shows the unit costs (in 1999 dollars) — that is, estimates of the average cost 
of providing specified services to one person for a month — used in the cost analysis. Differ- 
ences in unit measures make it difficult to compare all unit costs across all programs, but it is 
clear from those with common units that costs varied widely across programs. These differences 
reflect a number of factors: differences in local markets, the types of employment-related services 
that participants selected or were steered toward, and the decisions that administrators made in 
implementing their programs. For example, some programs referred sample members to commu- 
nity providers for basic education services, but did not provide any additional fimding to these 
generally publicly funded programs. However, the welfare departments in Riverside, Detroit, and 
Portland paid for services provided to sample members through formal contracts with education 
providers. As a result, welfare department unit costs for basic education (column 1) are higher in 
these three sites than in the others.‘* Non-welfare agency unit costs for basic education (column 3) 



*A unit cost for basic education in Detroit is not available. Table 13.2 presents an “education and training” unit 
cost for Detroit, which combines vocational training, post-secondary education, and basic education. (See footnote 
“e” on Table 13.2.) 
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National Evaluation of Welfare-to-Work Strategies 
Table 13.2 

Estimated Unit Costs for Employment-Related Activities (in 1999 dollars) 



Program Group 




Control Group 


Welfare 


Non-Welfare 


Non-Welfare 


Department Unit Cost 


Agency Unit 


Agency Unit Cost 



Average Average 

per Month of Average per Average per per Month of Average per 
Program and Activity Participation ($) Session ($) Hour ($) Participation ($) Hour ($) 



Atlanta Labor Force Attachment 


Orientation and appraisal 


n/a 


72 


n/a 


n/a 


n/a 


Formal assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


Job search 


416 


n/a 


n/a 


61 


n/a 


Basic education 


115 


n/a 


3.18 


n/a 


3.20 


Post-secondary education 


73 


n/a 


9.03 


n/a 


8.44 


Vocational training 


153 


n/a 


7.31 


n/a 


6.88 


Work experience 


167 


n/a 


n/a 


185 


n/a 


Atlanta Human Capital Development 


Orientation and appraisal 


n/a 


72 


n/a 


n/a 


n/a 


Formal assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


Job search 


463 


n/a 


n/a 


61 


n/a 


Basic education 


99 


n/a 


3.09 


n/a 


3.20 


Post-secondary education 


94 


n/a 


8.07 


n/a 


8.44 


Vocational training 


140 


n/a 


7.25 


n/a 


6.88 


Work experience 


203 


n/a 


n/a 


185 


n/a 


Grand Rapids Labor Force Attachment 


Orientation and appraisal 


n/a 


18 


n/a 


n/a 


n/a 


Formal assessment 


395 


n/a 


n/a 


n/a 


n/a 


Job search^ 


259 


n/a 


n/a 


n/a 


n/a 


Basic education 


132 


n/a 


6.37 


n/a 


6.38 


Post-secondary education 


97 


n/a 


9.31 


n/a 


9.37 


Vocational training 


no 


n/a 


7.82 


n/a 


7.97 


Work experience 


241 


n/a 


n/a 


241 


n/a 


Grand Rapids Human Capital Development 


Orientation and appraisal 


n/a 


18 


n/a 


1 

n/a 


n/a 


Formal assessment 


395 


n/a 


n/a 


n/a 


n/a 


Job search^ 


259 


n/a 


n/a 


n/a 


n/a 


Basic education 


132 


n/a 


6.31 


n/a 


6.38 


Post-secondary education 


97 


n/a 


9.15 


n/a 


9.37 


Vocational training 


no 


n/a 


7.23 


n/a 


7.97 


Work experience 


241 


n/a 


n/a 


241 


n/a 












(continued) 
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Table 13.2 (continued) 





Program Group 




Control Group 






Welfare 

Department Unit Cost 


Non-Welfare 
Agency Unit 


Non-Welfare 
Agency Unit Cost 




Average 
per Month of 

Program and Activity Participation ($) 


Average per 
Session ($) 


Average per 
Hour ($) 


Average 

per Month of Average per 
Participation ($) Hour ($) 


Riverside Labor Force Attachment 


Orientation and appraisal 


n/a 


88 


n/a 


n/a 


n/a 


Formal assessment 


n/a 


596 


n/a 


n/a 


n/a 


Job search 


759 


n/a 


n/a 


254 


n/a 


Basic education 


255 


n/a 


4.45 


n/a 


4.09 


Post-secondary education 


122 


n/a 


6.37 


n/a 


6.39 


Vocational training 


122 


n/a 


5.52 


n/a 


5.90 


Work experience 


572 


n/a 


n/a 


572 


n/a 


Riverside Human Capital Development 
(without a high school diploma or G£D) 

Orientation and appraisal 


n/a 


88 


n/a 


n/a 


n/a 


Formal assessment 


n/a 


596 


n/a 


n/a 


n/a 


Job search 


759 


n/a 


n/a 


254 


n/a 


Basic education 


255 


n/a 


4.05 


n/a 


4.09 


Post-secondary education 


122 


n/a 


6.29 


n/a 


6.39 


Vocational training 


122 


n/a 


5.31 


n/a 


5.90 


Work experience 


572 


n/a 


n/a 


572 


n/a 


Columbus Integrated 


Orientation and appraisal 


19 


n/a 


n/a 


n/a 


n/a 


Formal assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


Job search 


220 


n/a 


n/a 


100 


n/a 


Basic education 


86 


n/a 


3.82 


n/a 


n/a 


Post-secondary education 


225 


n/a 


6.16 


n/a 


n/a 


Vocational training^ 


225 


n/a 


n/a 


n/a 


n/a 


Work experience 


106 


n/a 


n/a 


76 


n/a 


Columbus Traditional 


Orientation and appraisal 


14 


n/a 


n/a 


n/a 


n/a 


Formal assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


Job search 


205 


n/a 


n/a 


100 


n/a 


Basic education 


54 


n/a 


3.99 


n/a 


n/a 


Post-secondary education 


55 


n/a 


6.16 


n/a 


n/a 


Vocational training^ 


55 


n/a 


n/a 


n/a 


n/a 


Work experience 


46 


n/a 


n/a 


76 


n/a 


Detroit‘S 


Orientation and appraisal 


n/a 


n/a 


n/a 


n/a 


n/a 


Formal assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


Job search^ 


752 


n/a 


n/a 


597 


n/a 


Education and training® 


587 


n/a 


8.44 


n/a 


8.21 


Work experience 


557 


n/a 


n/a 


n/a 


n/a 



(continued) 
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Table 13.2 (continued) 



Program Group 




Control Group 


Welfare 


Non-Welfare 


Non-Welfare 


Department Unit Cost 


Agency Unit 


Agency Unit Cost- 



Average Average 

per Month of Average per Average per per Month of Average per 



Program and Activity Participation ($) Session ($) Hour ($) Participation ($) Hour ($) 



Portland 

Orientation and appraisal 


138 


n/a 


n/a 


n/a 


n/a 


Formal assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


Job search 


359 


n/a 


n/a 


280 


n/a 


Basic education 


257 


n/a 


11.75 


n/a 


11.75 


Post-secondary education 


131 


n/a 


7.82 


n/a 


8.01 


Vocational training 


131 


n/a 


7.82 


n/a 


8.01 


Work experience 


400 


n/a 


n/a 


322 


n/a 



SOURCES: MDRC calculations based on fiscal and participation data from the following: Atlanta - the Fulton County 
Department of Family and Children Services, the Georgia Department of Human Resources, the Georgia Department of 
Technical and Adult Education, the Board of Regents University System of Georgia; Grand Rapids - the Michigan 
Department of Social Services, the Michigan Department of Education Office of Extended Learning Services, the Grand 
Rapids Community College, the Wyoming Community Education Center; Riverside - the California Department of 
Social Services, the California Department of Education, the Chancellor’s Office of California Community Colleges; 
Columbus - the Franklin County Department of Human Services, the Ohio Department of Education, the Office of 
Vocational and Adult Education, the Ohio Board of Regents, the National Center for Education Statistics; Detroit - the 
Michigan Family Independence Agency, the Michigan Department of Education, the Michigan Jobs Commission; 
Portland - the Oregon Department of Human Resources, Adult and Family Services Division, the Oregon Office of 
Community College Services; in all sites - information collected on tuition charged at proprietary schools attended by 
sample members, from MDRC-collected JOBS case file data, the MDRC Two-Year Client Survey, and the MDRC Five- 
Year Client Survey. 

NOTES: Welfare department unit costs include the cost of providing activities (e.g., classroom instruction, job search 
facilitation, space rental, and case management expenditures). 

Work experience unit costs for program and control group members receiving services from non-welfare agencies 
were assumed to be equal to the welfare department JOBS unit cost. Atlanta’s unit cost for controls was the average of 
the LFA and HCD welfare department unit costs. 

N/a = not applicable. 

^he estimated unit cost of job search to Grand Rapids non- welfare agencies was $1,025 per participant. 

*^The estimated unit cost of vocational training for non- welfare agencies in Columbus was $5,032 per participant in 
the Integrated program and $2,771 per participant in the Traditional program. 

^For the Detroit program, welfare department unit costs were calculated by dividing total activity expenditures by 
the total number of participant-months for the corresponding activity. At the time of the two-year follow-up, Farrell, 
2000, presented a range of unit costs, based on two different methods of calculating participant-months. However, both 
the two- and five-year gross cost estimates were based only on the upper-bound unit cost estimates, and so only ’’upper- 
bound” estimates of unit costs are presented on this table. See Farrell, 2000, for a complete discussion of this issue and 
the full range of two-year cost and participation estimates for this site. 

^he estimated unit cost of job search to Detroit non- welfare agencies was $816 per month of participation. In 
addition, the estimated unit cost to the welfare department for control group participation in job search, education and 
training, and work experience activities was $271 per month of participation. 

^Owing to data limitations, in Detroit it was not possible to separate vocational training, post-secondary education, 
and basic education costs. Thus, the gross costs of all three activities are included in the cost of ’’education and training.” 
Furthermore, in this site orientation and assessment costs are spread across these three activities. 
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were much higher for the Portland program than for the other programs, because community col- 
leges were the largest provider of basic education services in Portland, whereas public adult edu- 
cation systems were the main providers of basic education in the other sites. 

The unit cost multiplied by sample members’ average number of months of participation 
in the activity (the behavioral variable) yields the cost per sample member. These costs are 
shown in Table 13.3 (column 1). In-program welfare department operating costs ranged from 
$603 (Columbus Traditional) to $2,753 (Riverside HCD) and were not systematically different 
between employment- and education-focused programs. 

2. Support service costs. Programs paid for child care, transportation, and ancillary 
services (such as uniforms, tools, equipment, and books) to support recipients’ participation in em- 
ployment and employment-related activities. Detailed information on support service payments was 
available through the end year 2 of follow-up. These data, together with data from the Five-Year 
Client Survey, were used to estimate payments for the remainder of the follow-up period. 

As shown in Table 13.3, the welfare departments spent an average of $252 (Riverside 
LFA) to more than $1,000 (Atlanta LFA and HCD and Portland) per program group member on 
in-program child care. The costs of other support services, which included payments to cover 
transportation costs, class fees, uniforms, and so on, ranged from less than $100 (Grand Rapids 
LFA and HCD) to $610 (Atlanta HCD). 

B. Non-Welfare Agency Expenditures 

Table 13.3 (column 2) shows the non-welfare agency cost of providing employment- 
related services. These costs ranged from $1,680 (Portland) to $5,288 (Grand Rapids HCD). As 
discussed above, Portland’s low non-welfare agency costs reflect the fact that the welfare de- 
partment paid for the majority of in-program activities (except for vocational training and col- 
lege). The costs were high in Grand Rapids, especially for the HCD program, because job search 
services were provided and paid for by the local community education center and because of the 
extensive use of education and training activities, which also were not paid for by the welfare 
department. 

V. Expenditures for Out-of-Program Employment-Related Services 

Program and control group members participated in some employment-related activities 
when they were not receiving welfare benefits. Although these services were not part of the 
NEWWS programs, if program and control group members participated in such activities and par- 
ticipated at different rates, the off- welfare services have the potential to differentially increase sam- 
ple members’ earnings and reduce their welfare receipt, and thus are included in the cost estimates. 
These expenditures are examined in more detail below. (See Figure 13.1, Boxes 4 and 5.) 

A. Welfare Department Expenditures 

The bulk of the costs incurred when participants were not receiving welfare were paid by 
non-welfare agencies; welfare agencies provided child care assistance only. Sample members 
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National Evaluation of Welfare-to-Work Strategies 
Table 13.3 

Estimated Cost per Program Group Member for Employment-Related Services 
Witbin a Five-Year Follow-Up Period, 
by Program, Service Component, and Agency (in 1999 dollars) 





In-Program Cost 




Out-of-Program Cost 


Total Gross 
Cost per 
^am Group 
Member ($) 


Activity or Service 


Welfare Non-Welfare 
D ep artment Agency 

Cost ($) Cost ($) 


Total Welfare Non-Welfare 

Program Department Agency Pro| 

Cost($) Cost($) Cost($) ] 


Atlanta Labor Force Attachment 












Orientation and appraisal 


72 


0 


72 


0 


0 


72 


Formal assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


n/a 


Job search 


1,081 


0 


1,081 


0 


23 


1,104 


Basic education 


210 


453 


663 


0 


166 


829 


Post-secondary education 


62 


632 


694 


0 


177 


871 


Vocational training 


265 


1,166 


1,431 


0 


641 


2,072 


Work experience 


256 


n/a 


256 


0 


67 


323 


Subtotal (operating) 


1,946 


2,251 


4,198 


0 


1,074 


5,271 


Child care 


1,191 


0 


1,191 


.301 


0 


1,491 


Other support services 


317 


0 


317 


0 


0 


317 


Total 


3,454 


2,251 


5,705 


301 


1,074 


7,080 


Atlanta Human Capital Development 












Orientation and appraisal 


72 


0 


72 


0 


0 


72 


Formal assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


n/a 


Job search 


503 


0 


503 


0 


28 


530 


Basic education 


430 


1,004 


1,434 


0 


161 


1,596 


Post-secondary education 


58 


425 


483 


0 


415 


898 


Vocational training 


429 


2,185 


2,614 


0 


605 


3,219 


Work experience 


216 


0 


216 


0 


45 


261 


Subtotal (operating) 


1,708 


3,614 


5,323 


0 


1,253 


6,576 


Child care 


1,148 


0 


1,148 


251 


0 


1,400 


Other support services 


610 


0 


610 


0 


0 


610 


Total 


3,466 


3,614 


7,080 


251 


1,253 


8,585 


Grand Rapids Labor Force Attachment 












Orientation and appraisal 


18 


n/a 


18 


0 


0 


18 


Formal assessment 


9 


n/a 


9 


0 


0 


9 


Job search 


362 


908 


1,270 


0 


152 


1,422 


Basic education 


166 


719 


884 


0 


484 


1,368 


Post-secondary education 


263 


1,710 


1,974 


0 


1,340 


3,314 


Vocational training 


152 


1,033 


1,186 


0 


618 


1,804 


Work experience 


130 


0 


130 


0 


53 


183 


Subtotal (operating) 


1,100 


4,371 


5,470 


0 


2,648 


8,118 


Child care 


451 


0 


■451 


312 


0 


762 


Other support services 


60 


0 


60 


0 


0 


60 


Total 


1,610 


4,371 


5,981 


312 


2,648 


8,940 
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Table 13.3 (continued) 



Activity or Service 




In-Program Cost 




Out-of-Program Cost 


Total Gross 
Cost per 
Program Group 
Member ($) 


Welfare 
Department 
Cost ($) 


Non-Welfare 
Agency 
Cost ($) 


Total 
Program 
Cost ($) 


Welfare 
Department 
Cost ($) 


Non-Welfare 
Agency 
Cost ($) 


Grand Rapids Human Capital Development 










Orientation and appraisal 


18 


0 


18 


0 


0 


18 


Formal assessment 


312 


0 


312 


0 


0 


312 


Job search 


206 


432 


638 


0 


116 


754 


Basic education 


474 


1,672 


2,146 


0 


443 


2,589 


Post-secondary education 


302 


2,285 


2,586 


0 


995 


3,582 


Vocational training 


163 


900 


1,062 


0 


956 


2,018 


Work experience 


131 


0 


131 


0 


139 


271 


Subtotal (operating) 


1,607 


5,288 


6,895 


0 


2,649 


9,544 


Child care 


606 


0 


606 


502 


0 


1,108 


Other support services 


75 


0 


75 


0 


0 


75 


Total 


2,288 


5,288 


7,576 


502 


2,649 


10,727 


Riverside Labor Force Attachment 












Orientation and appraisal 


111 


0 


111 


0 


0 


111 


Formal assessment 


6 


0 


6 


0 


0 


6 


Job search 


1,056 


0 


1,056 


0 


145 


1,202 


Basic education 


157 


345 


502 


0 


226 


728 


Post-secondary education 


218 


988 


1,205 


0 


984 


2,189 


Vocational training 


97 


392 


489 


0 


586 


1,075 


Work experience 


147 


0 


147 


0 


134 


281 


Subtotal (operating) 


1,793 


1,725 


3,518 


0 


2,075 


5,593 


Child care 


252 


0 


252 


29 


0 


281 


Other support services 


171 


0 


171 


0 


0 


171 


Total 


2,217 


1,725 


3,941 


29 


2,075 


6,045 


Riverside Human Capital Development 












(without a high school diploma or GED) 












Orientation and appraisal 


107 


0 


107 


0 


0 


107 


Formal assessment 


13 


0 


13 


0 


0 


13 


Job search 


1,008 


0 


1,008 


0 


169 


1,177 


Basic education 


1,188 


1,420 


2,608 


0 


291 


2,900 


Post-secondary education 


89 


209 


298 


0 


282 


580 


Vocational training 


179 


688 


867 


0 


182 


1,049 


Work experience 


169 


0 


169 


0 


92 


261 


Subtotal (operating) 


2,753 


2,318 


5,071 


0 


1,015 


6,086 


Child care 


361 


0 


361 


16 


0 


376 


Other support services 


170 


0 


170 


0 


0 


170 


Total 


3,284 


2,318 


5,601 


16 


1,015 


6,632 



(continued) 
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Table 13.3 (continued) 



Activity or Service 


In-Program Cost 




Out-of-Program Cost 


Total Gross 
Cost per 
Program Group 
Member ($) 


Welfare 
Department 
Cost ($) 


Non- Welfare 
Agency 
Cost ($) 


Total 
Program 
Cost ($) 


Welfare 
Department 
Cost ($) 


Non-Welfare 
Agency 
Cost ($) 


Columbus Integrated 














Orientation and appraisal 


18 


0 


18 


0 


0 


18 


Formal assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


n/a 


Job search 


141 


0 


141 


0 


15 


156 


Basic education 


108 


259 


367 


0 


334 


700 


Post-secondary education 


732 


1,241 


1,973 


0 


887 


2,860 


Vocational training 


206 


542 


749 


0 


534 


1,283 


Work experience 


85 


0 


85 


0 


170 


254 


Subtotal (operating) 


1,290 


2,042 


3,331 


0 


1,939 


5,271 


Child care 


571 


0 


571 


558 


0 


1,129 


Other support services 


334 


0 


334 


0 


0 


334 


Total 


2,195 


2,042 


4,237 


558 


1,939 


6,734 


Columbus Traditional 














Orientation and appraisal 


9 


0 


9 


0 


0 


9 


Formal assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


n/a 


Job search 


192 


0 


192 


0 


0 


192 


Basic education 


84 


494 


577 


0 


218 


796 


Post-secondary education 


194 


1,870 


2,064 


0 


556 


2,620 


Vocational training 


59 


270 


329 


0 


830 


1,158 


Work experience 


66 


0 


66 


0 


127 


194 


Subtotal (operating) 


603 


2,634 


3,237 


0 


1,731 


4,969 


Child care 


584 


0 


584 


547 


0 


1,131 


Other support services 


224 


0 


224 


0 


0 


224 


Total 


. 1,411 


2,634 


4,045 


547 


1,731 


6,323 


Detroit 














Job search 


114 


446 


560 


0 


5 


565 


Education and training® 


1,198 


1,403 


2,602 


0 


1,879 


4,481 


Work experience 


77 


0 


77 


0 


3 


80 


Subtotal (operating) 


1,389 


1,849 


3,238 


0 


1,887 


5,126 


Child care 


313 


0 


313 


435 


0 


748 


Other support services 


117 


0 


117 


0 


0 


117 


Total 


1,819 


1,849 


3,668 


435 


1,887 


5,990 


(continued) 
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Table 13.3 (continued) 



Activity or Service 




In-Program Cost 




Out-of-Program Cost 


Total Gross 
Cost per 
Program Group 
Member ($) 


Welfare 
Department 
Cost ($) 


Non-Welfare 
Agency 
Cost ($) 


Total 
Program 
Cost ($) 


Welfare 
Department 
Cost ($) 


Non-Welfare 
Agency 
Cost ($) 


Portland 














Orientation and appraisal 


138 


11 


149 


0 


0 


149 


Formal assessment 


n/a 


n/a 




n/a 


n/a 




Job search 


609 


14 


623 


0 


131 


754 


Basic education 


361 


12 


373 


0 


624 


997 


Post-secondary education 


127 


709 


836 


0 


755 


1,591 


Vocational training 


182 


928 


1,110 


0 


831 


1,941 


Work experience 


286 


6 


292 


.0 


122 


414 


Subtotal (operating) 


1,702 


1,680 


3,382 


0 


2,464 


5,846 


Child care 


1,043 


0 


1,043 


1,883 


0 


2,926 


Other support services 


158 


0 


158 


0 


0 


159 


Total 


2,903 


1,680 


4,583 


1,883 


2,464 


8,930 



SOURCES: MDRC calculations based on fiscal and participation data from the following: Atlanta - the Fulton County 
Department of Family and Children Services, the Georgia Department of Human Resources, the Georgia Department 
of Technical and Adult Education, the Board of Regents University System of Georgia; Grand Rapids - the Michigan 
Department of Social Services, the Michigan Department of Education Office of Extended Learning Services, the 
Grand Rapids Community College, the Wyoming Community Education Center; Riverside - the California Department 
of Social Services, the California Department of Education, the Chancellor’s Office of California Community Colleges; 
Columbus - the Franklin County Department of Human Services, the Ohio Department of Education, the Office of 
Vocational and Adult Education, the Ohio Board of Regents, the National Center for Education Statistics; Detroit - the 
Michigan Family Independence Agency, the Michigan Department of Education, the Michigan Jobs Commission; 
Portland - the Oregon Department of Human Resources, Adult and Family Services Division, the Oregon Office of 
Community College Services; in all sites - information collected on tuition charged at proprietary schools attended by 
sanple members, from MDRC-collected JOBS case file data, the MDRC Two-Year Client Survey, and the MDRC 
Five-Year Client Survey. MDRC child care and other support service calculations from Fulton County, Kent County, 
Riverside County, Ohio Department of Human Services, Wayne County, Washington County, and Multnomah County 
(District 2) payment data. Other support service data from county records. 

NOTES: Rounding may cause slight discrepancies in calculating sums and differences. 

N/a = not applicable. 

“Owing to data limitations, in Detroit it was not possible to separate vocational training, post-secondary education, 
and basic education costs. Thus, the gross costs of all three activities are included in the cost of "education and 
training." Furthermore, in this site orientation and assessment costs are spread across these three activities. 
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were eligible for one year of transitional child care assistance if they left welfare for work and 
could also receive income-eligible child care. 

As shown in Table 13.3, the average cost of transitional, income-eligible, and other 
child care ranged from less than $30 per program group member (Riverside LFA and HCD) to 
$1,883 (Portland). This wide variation in child care payments was seen after two years of fol- 
low-up. As explained in the earlier report, Portland’s high child care costs may reflect a com- 
bination of the following: The program included more families with younger children than 
most of the other NEWWS programs,^ the staff believed in the importance of providing child 
care to everyone who was eligible for it, and integrated case management allowed better ad- 
ministration of child care payments.® 

B. Non-Welfare Agency Expenditures 

When sample members were not receiving welfare benefits, they most commonly partici- 
pated in basic education, post-secondary education, and vocational training; some sample mem- 
bers participated in other activities (job search, unpaid work experience, and on-the-job train- 
ing).^ These costs ranged from about $1,000 (Riverside HCD) to about $2,500 (Grand Rapids 
LFA and HCD and Portland). The programs in Grand Rapids and Portland had high rates of self- 
initiated participation, particularly in education and training activities. As will be discussed later, 
this was also true for control group members in these programs. 

VI. Gross Costs 

The gross costs were obtained by adding the cost of in-program services to the out-of- 
program costs. This total investment must be compared with the total gross cost per control 
group member to determine the government’s net investment per program group member and, in 
the benefit-cost analysis, the net payoff of that investment. As shown in Table 13.3, the gross 
costs of the NEWWS programs ranged from $5,990 (Detroit) to $10,727 (Grand Rapids HCD) 
per program group member over the five-year follow-up period. On average, gross costs were 
slightly higher for education-focused programs than for employment-focused programs. (See Fig- 
ure 13.1, Boxes 7 and 10.) 

A. Gross Costs for Program Group Members 

Participation rates and duration of participation, as well as whether participation took 
place while sample members were on or off welfare, determined both overall costs and the pro- 
portion of costs paid for by the welfare department. For example, post-secondary education unit 

^The Portland, Grand Rapids, and Detroit programs required welfare recipients with children as young as age 1 
to participate in the NEWWS program, whereas the requirement extended only to recipients with children as young 
as age 3 in the other programs. The Grand Rapids and Detroit programs did not have particularly high child care 
costs. 

*A comparison of child care costs for the Columbus Integrated and Traditional programs does not support the 
idea that the case management approach alone was responsible for this result. 

^The Two-Year and Five-Year Client Survey data, along with welfare payment records, were used to estimate 
participation in employment-related activities that sample members took part in when they were not receiving 
welfare. 
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costs were similar in the Atlanta and Grand Rapids HCD programs, but participation in both in- 
program and out-of-program (that is, self-initiated) activities was much greater in Grand Rapids, 
resulting in a total gross cost for post-secondary education that was nearly four times as high as 
that for Atlanta. As a result, the Grand Rapids HCD program incurred the highest in-program 
costs (column 3) and total costs (column 6). However, the proportion of these costs paid by the 
welfare department was lower in the Grand Rapids programs than in the other programs (column 
1 divided by column 6): These programs had more participation in higher-cost activities such as 
education and training that was leveraged with fewer dollars by the welfare department. 

Overall, higher costs were associated with higher rates of participation in education and 
training activities. Therefore, employment-focused programs might be expected to have lower 
total gross costs than education-focused programs. However, the gross costs of the Grand Rapids 
LFA and Portland programs were among the highest in this evaluation because sample members 
in these programs participated in education and training activities at high rates, much of it self- 
initiated. As a result, on average, employment-focused programs were slightly more expensive 
than education- focused ones. 

The average gross cost was higher than costs found in other MDRC evaluations of wel- 
fare-to-work programs. On average, total gross costs of the NEWWS programs were similar to 
those found in other high-cost programs, including the Alameda and Los Angeles GAIN pro- 
grams operated in the late 1980s and early 1990s, which provided extensive education and train- 
ing services. The average cost per sample member across all NEWWS programs was $7,599, 
compared with $7,763 in Alameda and $7,123 in Los Angeles (all in 1999 dollars).^ The higher 
costs in NEWWS programs (Atlanta and Grand Rapids HCD, Grand Rapids LFA, and Portland) 
are likely due to higher participation overall, and particularly in high-cost activities such as post- 
secondary education and vocational training. 

B. Gross Costs for Control Group Members 

Control group costs reflect the cost of participation in employment-related services, most 
of which was self- initiated. However, in the sites where control embargoes were lifted before the 
end of the five-year follow-up period, a small portion of participation was program-related.^ 

In general, control group costs were high. This was particularly true in Grand Rapids and 
Portland owing to high rates of self-initiated participation, especially in post-secondaiy education 
and vocational training activities. The average control group cost in the GAIN program was 
$1,638 (in 1999 dollars). The control group costs in Grand Rapids and Portland were about three 
times higher than the highest control group cost in the GAIN program, which was $2,233 (in 
1999 dollars) in San Diego. The control group costs in the other NEWWS programs ranged from 
8 percent (Riverside HCD) to 76 percent (Detroit) higher. 



*In Riccio, Friedlander, and Freedman, 1994, the GAIN costs were presented in 1993 dollars and were as fol- 
lows: Alameda, $6,977; and Los Angeles, $6,402. 

*A portion of job search participation that occurred after the embargoes were lifted was considered program- 
related. The proportion of participation that was considered in-program was estimated based on the number of 
months that sample members received welfare in the post-embargo period. All participation in education and training 
activities was considered to be self-initiated. 




- 313 - 



358 



VII. Net Costs 



The net cost per program group member is the gross cost per program group member mi- 
nus what would have been spent in the absence of the NEWWS programs, represented by the 
gross cost per control group member. (See Figure 13.1, Box 11.) As shown in Table 13.4 (col- 
umn 3), net costs ranged from about $2,000 (Grand Rapids LFA and Detroit) to $5,480 (Atlanta 
HCD). Depending on the rate of participation, and the types of activities in which control group 
members enroll on their own, programs with high gross costs can have low net costs. For exam- 
ple, although Grand Rapids HCD had the highest gross costs, owing to high participation rates in 
education and training activities, similarly high participation rates in these activities among con- 
trol group members resulted in a net cost that was lower than costs for the other education- 
focused programs. Likewise, Grand Rapids LFA and Portland had the highest gross costs among 
the employment-focused programs, but high gross costs for control group members produced low 
net costs for these two programs. As a result, the average net cost of the NEWWS programs was 
similar to that of the GAIN program. 

Net costs were generally lower for employment-focused programs than for education- 
focused ones, largely due to the particularly low net costs in Grand Rapids LFA and Portland. As 
the two-year findings suggested, the Grand Rapids LFA program may have diverted program 
group members into job search who would have enrolled on their own in education or training 
programs. The same may have been true of the Portland program. 

A. Results by Educational Attainment Sub2roups 

Table 13.5 shows gross and net costs for those with and without a high school diploma or 
GED when they entered the evaluation. For the most part, gross and net costs were higher for 
graduates. This is not surprising, given that those with high school credentials were more likely 
to participate in higher-cost activities, such as post-secondary education or vocational training, 
whereas those who lacked these credentials were more likely to participate in typically lower-cost 
activities, such as basic education. 

B. LFA-HCD Program Differences 

As noted above, net costs were generally higher for education-focused programs than for 
employment-focused programs. Similarly, HCD programs had higher net costs than LFA pro- 
grams. Again, this was not a surprising finding. As discussed in Chapter 1, the HCD programs 
were designed to make larger upfront investments in building skills, mainly through education 
and training activities, which tend to be longer in duration and more expensive to operate than 
job search activities. It was anticipated that the programs would incur higher costs, but with the 
hope that this greater investment in human capital would produce greater returns (earnings in- 
creases and welfare savings) over time. 

After two years of follow-up, HCD programs were 1.5 to 2.6 times more expensive than 
LFA programs. Although the net cost gap at the five-year point narrowed relative to net costs at 



- 314 - 



359 



National Evaluation of Welfare-to-Work Strategies 
Table 13.4 



Estimated Gross and Net Costs Within a Five-Year Follow-Up Period, 
by Program and Service Component (in 1999 dollars) 



Program and Component 


Total Gross 
Cost per 
Program Group 
Member ($) 


Total Gross 
Cost per 
Control Group 
Member ($) 


Total Net 
Cost per 
Program Group 
Member ($) 


Atlanta Labor Force Attachment 








Orientation and appraisal 


72 


11 


61 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


1,104 


79 


1,025 


Basic education 


829 


166 


663 


Post-secondary education 


871 


844 


27 


Vocational training 


2,072 


1,392 


680 


Work experience 


323 


126 


198 


Subtotal (operating) 


5,271 


2,618 


2,654 


Child care 


1,491 


459 


1,032 


Other support services 


317 


29 


288 


Total 


7,080 


3,105 


3,974 


Atlanta Human Capital Development 








Orientation and appraisal 


72 


11 


61 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


530 


79 


452 


Basic education 


1,596 


166 


1,429 


Post-secondary education 


898 


844 


54 


Vocational training 


3,219 


1,392 


1,827 


Work experience 


261 


126 


135 


Subtotal (operating) 


6,576 


2,618 


3,958 


Child care 


1,400 


459 


940 


Other support services 


610 


29 


581 


Total 


8,585 


3,105 


5,480 


Grand Rapids Labor Force Attachment 






Orientation and appraisal 


18 


1 


17 


Formal assessment 


9 


n/a 


9 


Job search 


1,422 


329 


1,093 


Basic education 


1,368 


1,170 


198 


Post-secondary education 


3,314 


2,906 


408 


Vocational training 


1,804 


1,931 


-127 


Work experience 


183 


54 


129 


Subtotal (operating) 


8,118 


6,392 


1,726 


Child care 


762 


561 


202 


Other support services 


60 


n/a 


60 


Total 


8,940 


6,953 


1,987 



(continued) 
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Table 13.4 (continued) 





Total Gross 


Total Gross 


Total Net 




Cost per 


Cost per 


Cost per 


Program Group 


Control Group 


Program Group 


Program and Component 


Member ($) 


Member ($) 


Member ($) 


Grand Rapids Human Capital Development 






Orientation and appraisal 


18 


1 


17 


Formal assessment 


312 


n/a 


312 


Job search 


754 


329 


425 


Basic education 


2,589 


1,170 


1,419 


Post-secondary education 


3,582 


2,906 


675 


Vocational training 


2,018 


1,931 


87 


Work experience 


271 


54 


216 


Subtotal (operating) 


9,544 


6,392 


3,151 


Child care 


1,108 


561 


548 


Other support services 


75 


n/a 


75 


Total 


10,727 


6,953 


3,773 


Riverside Labor Force Attachment 








Orientation and appraisal 


111 


n/a 


111 


Formal assessment 


6 


n/a 


6 


Job search 


1,202 


149 


1,053 


Basic education 


728 


230 


498 


Post-secondary education 


2,189 


1,301 


889 


Vocational training 


1,075 


856 


218 


Work experience 


281 


126 


155 


Subtotal (operating) 


5,593 


2,661 


2,931 


Child care 


281 


63 


218 


Other support services 


171 


n/a 


171 


Total 


6,045 


2,724 


3,320 


Riverside Human Capital Development 
(without a high school diploma or G£D) 








Orientation and appraisal 


107 


n/a 


107 


Formal assessment 


13 


n/a 


13 


Job search 


1,177 


127 


1,050 


Basic education 


2,900 


409 


2,491 


Post-secondary education 


580 


606 


-26 


Vocational training 


1,049 


820 


229 


Work experience 


261 


63 


198 


Subtotal (operating) 


6,086 


2,025 


4,061 


Child care 


376 


35 


342 


Other support services 


170 


n/a 


170 


Total 


6,632 


2,059 


4,573 



(continued) 
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Table 13.4 (continued) 



Program and Component 


Total Gross 
Cost per 
Program Group 
Member ($) 


Total Gross 
Cost per 
Control Group 
Member ($) 


Total Net 
Cost per 
Program Group 
Member ($) 


Columbus Integrated 








Orientation and appraisal 


18 


n/a 


18 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


156 


31 


125 


Basic education 


700 


180 


520 


Post-secondary education 


2,860 


606 


2,254 


Vocational training 


1,283 


593 


690 


Work experience 


254 


25 


229 


Subtotal (operating) 


5,271 


1,434 


3,837 


Child care 


1,129 


1,084 


45 


Other support services 


334 


33 


301 


Total 


6,734 


2,551 


4,183 


Columbus Traditional 








Orientation and appraisal 


9 


n/a 


9 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


192 


31 


161 


Basic education 


796 


180 


616 


Post-secondary education 


2,620 


606 


2,014 


Vocational training 


1,158 


593 


566 


Work experience 


194 


25 


169 


Subtotal (operating) 


4,969 


1,434 


3,534 


Child care 


1,131 


1,084 


47 


Other support services 


224 


33 


190 


Total 


6,323 


2,551 


3,772 


Detroit 








Job search 


565 


328 


237 


Education and training® 


4,481 


2,830 


1,651 


Work experience 


80 


12 


67 


Subtotal (operating) 


5,126 


3,170 


1,956 


Child care 


748 


734 


14 


Other support services 


117 


34 


83 


Total 


5,990 


3,937 


2,053 


(continued) 
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Table 13.4 (continued) 



Program and Component 


Total Gross 
Cost per 
Program Group 
Member ($) 


Total Gross 
Cost per 
Control Group 
Member ($) 


Total Net 
Cost per 
Program Group 
Member ($) 


Portland 








Orientation and appraisal 


149 


n/a 


149 


Formal assessment 


n/a 


n/a 


n/a 


Job search 


754 


273 


481 


Basic education 


997 


593 


404 


Post-secondary education 


1,591 


2,353 


-763 


Vocational training 


1,941 


1,049 


892 


Work experience 


414 


240 


174 


Subtotal (operating) 


5,846 


4,508 


1,337 


Child care 


2,926 


1,542 


1,384 


Other support services 


159 


15 


144 


Total 


8,930 


6,065 


2,865 



SOURCES: See Table 13.3. 

NOTES: Rounding may cause slight discrepancies in calculating sums and differences. 

N/a = not applicable. 

Owing to data limitations, in Detroit it was not possible to separate vocational training, post^secondary 
education, and basic education costs. Thus, the gross costs of all three activities are included in the cost of 
’’education and training.” Furthermore, in this site orientation and assessment costs are spread across these three 
activities. 
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National Evaluation of Welfare-to-Work Strategies 
Table 13.5 



Estimated Gross and Net Costs Within a Five-Year Follow-Up Period, 
by Program and Education Subgroup (in 1999 dollars) 



Total Gross 
Cost per 
Program Group 

Program and Component Member ($) 


Total Gross 
Cost per 
Control Group 
Member ($) 


Total Net 
Cost per 
Program Group 
Member ($) 


Without a high school dioloma or GED 








Atlanta Labor Force Attachment 


Operating costs 


3,761 


1,118 


2,642 


Support services 


1,469 


370 


1,099 


Total 


5,230 


1,489 


3,741 


Atlanta Human Capital Development 


Operating costs 


5,646 


1,118 


4,528 


Support services 


1,982 


370 


1,612 


Total 


7,628 


1,489 


6,140 


Grand Rapids Labor Force Attachment 


Operating costs 


6,244 


5,699 


544 


Support services 


493 


445 


48 


total 6,736 

Grand Rapids Human Capital Development 


6,144 


592 


Operating costs 


8,789 


5,699 


3,090 


Support services 


1,001 


445 


556 


Total 


9,790 


6,144 


3,645 


Riverside Labor Force Attachment 


Operating costs 


5,306 


2,025 


3,282 


Support services 


309 


35 


274 


Total 


5,615 


2,060 


3,555 


Riverside Human Capital Development 


Operating costs 


6,086 


2,025 


4,061 


Support services 


546 


35 


511 


total 


6,632 


2,060 


4,572 


Columbus Integrated 


Operating costs 


4,564 


914 


3,650 


Support services 


1,357 


624 


733 


total 


5,921 


1,538 


4,383 


Columbus Traditional 


Operating costs 


3,500 


914 


2,585 


Support services 


1,078 


624 


454 


Total 


4,577 


1,538 


3,039 


Detroit 


Operating costs 


4,560 


3,033 


1,527 


Support services 


781 


902 


-121 


total 


5,341 


3,935 


1,406 


Portland 


Operating costs 


4,690 


2,927 


1,763 


Support services 


2,874 


1,346 


1,528 


total 


7,563 


4,273 


3,290 



(continued) 
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Table 13.5 (continued) 

Total Gross Total Gross • Total Net 

Cost per Cost per Cost per 

Program Group Control Group Program Group 

Program and Component Member ($) Member ($) Member ($) 

With a high school diploma or GED 



Atlanta Labor Force Attachment 



Operating costs 


6,259 


3,596 


2,663 


Support services 


2,105 


559 


1,546 


Total 


8,365 


4,155 


4,209 


Atlanta Human Capital Development 


Operating costs 


7,207. 


3,596 


3,611 


Support services 


2,021 


559 


1,462 


Total 


9,228 


4,155 


5,073 


Grand Rapids Labor Force Attachment 


Operating costs 


8,961 


7,112 


1,849 


Support services 


958 


630 


328 


Total 


9,918 


7,741 


2,177 


Grand Rapids Human Capital Development 


Operating costs 


10,273 


7,112 


3,162 


Support services 


1,440 


630 


810 


Total 


11,713 


7,741 


3,972 


Riverside Labor Force Attachment 


Operating costs 


5,877 


3,625 


2,252 


Support services 


581 


58 


523 


Total 


6,457 


3,682 


2,775 


Columbus Integrated 


Operating costs 


5,938 


1,790 


4,149 


Support services 


1,524 


1,368 


157 


Total 


7,463 


3,157 


4,305 


Columbus Traditional 
Operating costs 


5,759 


1,790 


3,969 


Support services 


1,495 


1,368 


127 


Total 


7,253 


3,157 


4,096 


Detroit 


Operating costs 


5,510 


3,292 


2,218 


Support services 


859 


589 


270 


Total 


6,370 


3,882 


2,488 


Portland 


Operating costs 


6,568 


5,438 


1,131 


Support services 


3,358 


1,540 


1,818 


Total 


9,926 


6,978 


2,948 



SOURCES: See Table 13.3. 



NOTES: Rounding may cause slight discrepancies in calculating sums and differences. 
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the two-year point, HCD programs continued to be more expensive than LFA programs — from 
1.3 to 1.9 times more expensive.'*^ 

VIII. Analytical Approach for the Benefit-Cost Analysis 

The analytical approach used in this benefit-cost analysis is similar to that used in previ- 
ous evaluations conducted by MDRC.” The general approach is to place dollar values on the 
program’s effects and its use of resources whenever possible, either by directly measuring them 
or by estimating them. 

Program effects on earnings, welfare, and Food Stamp payments were measured directly. 
Effects on earnings were based on quarterly earnings reported by employers to states’ unemploy- 
ment insurance (UI) systems, and effects on welfare and Food Stamp payments were measured us- 
ing computerized administrative records (the same data sources were used in the impact analysis 
presented in Chapters 4 and 5). Effects on Medicaid, fringe benefits, federal income taxes, state in- 
come and sales taxes, and the costs of administering transfer programs could not be measured di- 
rectly, but were estimated or imputed using various data sources (details are provided below). 

All of these effects were considered along with the estimated net costs of the NEWWS 
programs, presented above, to ascertain the net gains and losses to program group members and 
to government budgets. 

A. Accounting Methods 

The benefit-cost estimates cover a five-year period starting with the quarter after random 
assignment (quarter 2) for each sample member. This time frame is similar to that used in most 
previous MDRC benefit-cost analyses of welfare-torwork programs. Projecting benefits and costs 
beyond five years would be problematic because it is difficult to predict future behavior. How- 
ever, given the extent to which impacts on earnings and welfare had decayed in most programs 
by the end of the five-year period, it seems unlikely that there would be many future net returns 
to the programs. 

The benefit-cost estimates in this analysis are expressed in terms of net present values per 
program group member. The “net” means that, like impacts, the estimated amounts represent dif- 
ferences between estimates for program and control group members. The estimates are in “pre- 
sent value” terms because the accounting method of discounting was used to express the dollar 
worth at the end of the first year of follow-up of program effects that occurred later in the follow- 



'°The Riverside comparison includes only HCD and LFA program group members without a high school di- 
ploma or GED at random assignment. 

"Many of the techniques were originally developed for the benefit-cost analysis conducted as part of the MDRC 
Demonstration of State Work/Welfare Initiatives (for additional information, see Long and Knox, 1985). In this re- 
port the description of the analytical approach was adapted from previous MDRC reports (Riccio, Friedlander, and 
Freedman, 1994; Kemple, Friedlander, and Fellerath, 1995; Miller et al., 2000; and Bloom et al., 2000a). 



12 * 

up period. As in the cost analysis, all estimated amounts in the benefit-cost analysis are ex- 
pressed in 1999 dollars, eliminating the effects of inflation on the values. 

B. Analytical Perspectives 

An important issue in benefit-cost analyses of government programs is determining who 
bears any benefits or costs of the program. A program effect can generate gains from one per- 
spective while generating losses from another. For example, a decrease in welfare payments is a 
financial loss from the perspective of the program group but a financial gain from the perspective 
of the government. This makes it important to consider the perspectives of all the directly af- 
fected groups when assessing each main program effect. 

This analysis presents the net benefits and costs of the NEWWS programs from the per- 
spective of program participants and government budgets. In the following table, the main finan- 
cial effects are shown as an expected gain or benefit (+), a loss or cost (-), or neither a benefit nor 
a cost (0), according to a priori expectations regarding their value. (The tables in the following 
sections show the actual gains and losses in dollars.) 



Expected Main Financial Effects 
of NEWWS Programs 


Accounting Perspective 


Welfare 

Sample 


Government 

Budget 


Increased earnings and fringe benefits 


+ 


0 


Increased tax payments 


- 


+ 


Reduced use of transfer programs 


- 


+ 


NEWWS operating costs 


0 


- 


Increased use of support services 


0 


- 



The welfare sample perspective identifies net gains or losses for program group members, 
indicating how they fared as a result of the program. As illustrated, it is expected that earnings 
impacts represent gains for participants, whereas reductions in welfare payments and higher tax 
payments (resulting from earnings gains) represent losses. The program may be considered a net 
gain from the standpoint of program group members if the gains from earnings exceed losses 



12 

In programs such as these, particularly education-focused programs, many costs are incurred early, when wel- 
fare receipt is heaviest; however, many benefits, such as earnings gains, continue to be realized in later years. Simply 
comparing the nominal dollar value of program costs with benefits over multiple years would be problematic because 
a dollar’s value is greater in the present than in the future: A dollar available today, to either sample members or the 
government, can be invested and may produce income over time, making it worth more than a dollar available in the 
future. In order to make a fair comparison between benefits and costs over multiple years, it is essential to determine 
their value at a common point in time — for example, the present. In benefit-cost analyses, this is often acconplished 
by discounting, a method for reducing the value of benefits and costs accrued in later years relative to benefits and 
costs accrued in early years. In this analysis, the end of the first year following random assignment was used as the 
comparison point for the investment period. In other words, gains that were accrued after that point were discounted 
to reflect their value at the end of year 1 . In calculating these discounted values, it was assumed that a dollar invested 
at the end of year 1 would earn a real rate of return of 5 percent annually (this assumption has been used in other 
MDRC benefit-cost analyses). 



from reduced transfer payments and higher taxes. The net cost of providing eligibility and em- 
ployment-related services to participants has no direct effect on their income. 

The government budget perspective identifies net gains and losses incurred by a combina- 
tion of federal, state, and local government budgets. Net gains to the government budget occur 
through savings in transfer payments and their related administrative costs and through higher 
taxes paid by program group members compared with control group members. The government 
budget comes out ahead if tax increases and savings in transfer payments and administrative 
costs exceed the net cost of providing NEWWS program services. Program group members’ 
earnings gains do not directly affect the government budget’s net gains or losses. 

C. Limitations of the Analysis 

Some limits on the comprehensiveness of the benefit-cost analysis should be recognized. 
Some program effects, whose costs and benefits are difficult to quantify or to express in dollars, are 
not estimated. For example, this analysis does not include estimates of out-of-pocket expenditures 
(for child care, transportation, clothing, and so on) that sample members incurred when they went to 
work.'^ In addition, the estimates in this chapter reflect the direct effects of programs and do not 
consider secondary effects. These secondary effects include the possible displacement of other 
workers by the increased employment of program group members; these displaced workers may 
have become unemployed or employed in lower-paying jobs. In addition, the analysis does not con- 
sider the sample members’ forgone personal and family activities that resulted from increased work 
or the intrinsic benefits of education that are not reflected in earnings. The analysis does not place a 
dollar value on family or child well-being or the clear but difficult-to-monetize benefits associated 
with society’s (or sample members’) preference for work over welfare. 

IX. Effects for Sample Members 

This section presents estimates of the financial benefits of the NEWWS programs per 
program group member during the five-year follow-up period. It presents an account of the 
main benefit components: earnings and fringe benefits, personal taxes, and transfer payments 
and benefits. 

A. Earnings and Fringe Benefits 

As reported in Chapter 4, the NEWWS programs produced increases in employment and 
earnings for program group members (compared with control group members) over the five-year 
follow-up period. Table 13.6 shows that the impact on earnings ranged from about $900 (Grand 
Rapids HCD) to more than $5,000 (Portland) per program group member. 

Fringe benefits — employer-provided health and life insurance, pension contributions, and 
worker’s compensation — were part of sample members’ total compensation from working and 



'^Some work-related expenditures were paid by the welfare agencies and are reflected as a cost to government 
budgets in the support services line. 

'"*The presentation of benefit-cost results in this report was adapted from previous MDRC reports (Miller et al., 
2000; Bloom et al., 2000a). 
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Table 13.6 

Estimated Program-Control Group Differences in 
Earnings, Fringe Benefits, and Personal Taxes per Sample Member 
Within a Five-Year Follow-Up Period (in 1999 dollars) 
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369 



estimate. Employee-paid Social Security and Medicare taxes are included as tax payments. 
^’Employee portion only. 



thus were included in the benefit-cost analysis.'^ They ranged by region from 13.7 percent (the 
West) to 15.8 percent (the Midwest) of earnings.'® Increases from earnings and fringe benefits 
yielded an average increase in total work-related compensation that ranged from $1,050 per pro- 
gram group member in Grand Rapids HCD to $5,879 per program group member in Portland.’’ 

B. Personal Taxes 

Since all of the programs increased earnings, one might expect them to also increase 
income taxes, payroll taxes, and sales and excise taxes (there is no sales tax in Oregon). Fed- 
eral tax payments, along with the Earned Income Tax Credit (EITC), were imputed from the 
relevant earnings base using tax rates and rules for 1995.'^ As shown in Table 13.6, in fact, 
about half of the programs decreased income taxes by $53 (Columbus Traditional) to $340 
(Riverside HCD) per sample member, fri these cases, the expected increase in tax payments 
was outweighed by an increase in federal EITC payments to sample members. There was an 
increase in federal income taxes in the remaining sites, ranging from less than $1 (Columbus 
Integrated) to $62 (Atlanta HCD). 

C. Transfer Payments and Benefits 

As discussed in Chapter 5, all of the programs in the evaluation decreased welfare and 
Food Stamp payments to program group members over the follow-up period. As shown in Table 
13.7, the impacts on welfare payments ranged from -$571 in the Detroit program to -$3,058 in 
the Riverside HCD program. During the same period, reductions in Food Stamp payments ranged 
from $160 in the Atlanta HCD program to over $1,000 in the Riverside HCD and Columbus In- 
tegrated programs. In sum, program group members received from about $900 (Detroit and At- 
lanta) to over $4,000 (Riverside HCD) less in cash payments over the five-year period than their 
control group counterparts. 

In addition to cash benefits, sample members were also eligible for Medicaid benefits. In 
low-grant states especially, Medicaid benefits can have a larger cash value than welfare pay- 
ments. Sample members and their children were categorically eligible for Medicaid while receiv- 
ing welfare. In addition, those who left welfare because of employment were eligible for Transi- 



^^Data limitations did not allow for an accounting of out-of-pocket costs paid by sample members for health care 
coverage. 

^^hese rates were based on published information on employers’ compensation costs for 1995 from the U.S. 
Department of Labor, Bureau of Labor Statistics. 

^^This analysis does not account for out-of-pocket child care costs. With the exception of the Portland program, 
no out-of-pocket cost differences were found between program and control group members in the programs for 
which Five-Year Client Survey data were available. 

^^The federal Earned Income Tax Credit is a credit against federal income taxes for taxpayers with annual earn- 
ings below a certain level. For 1995, taxpayers with earnings up to $26,673 were eligible for the EITC. Not all eligi- 
ble taxpayers receive the EITC. As has been the practice in earlier benefit-cost analyses performed by MDRC, the 
EITC “take-up” rate was estimated at 80 percent. 

* ^Income from earnings was used in calculating federal and state income taxes. Income from earnings and wel- 
fare benefits was used in calculating sales and excise taxes. Sales and excise tax rates were based on information 
from state tax boards. 
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Table 13.7 
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Tests of statistical significance were not performed. 



tional Medicaid for one year.^° Eligibility for these programs was estimated using welfare admin- 
istrative data, Transitional Medicaid take-up rates calculated from the Five-Year Client Survey,^' 
and published data on Medicaid expenditures.^^ 

As noted in Chapter 8, the programs in the NEWWS Evaluation did not result in signifi- 
cant differences in health care coverage between program and control group members at the end 
of the follow-up period. It is important to note that the earlier chapter looked at differences in 
health coverage at a single point in time, that is, in the month before the five-year survey. As 
shown in Figure 5.1, in most programs the smallest differences in rate of welfare receipt were 
observed at the end of the follow-up period. However, as shown in Table 5.1, the differences in 
months of welfare receipt (and, therefore, Medicaid coverage) over the entire five-year period 
were substantial, ranging from 1.6 months (Detroit) to 5.6 months (Portland). Although program 
group members were more likely than their control group counterparts to receive Transitional 
Medicaid, this did not make up for the loss in Medicaid benefits due to their shorter stay on wel- 
fare. As shown in Table 13.7, average Medicaid losses over the five-year period ranged from 
$309 (Detroit) to $2,598 (Portland). 

The costs of administering transfer programs were calculated as the cost per benefit paid. 
As a result, the NEWWS programs also decreased transfer program administration costs.^^ These 
changes, shown in Table 13.7, yielded decreases of between about $150 (Atlanta HCD and De- 
troit) and about $1,250 in Portland.^"^ 

X. Net Gains and Losses by Accounting Perspective 

A. Welfare Sample Members 

Table 13.8 summarizes the main financial effects from the perspective of welfare sample 
members. Program-control group differences were defined as gains, indicated by positive values, 
and losses, indicated by negative values. These results were then summed to attain an estimate of 
the overall net gain or loss of each program. 



^°This analysis does not include an accounting of public health care coverage for children through programs for 
low-income families, such as the Child Health Insurance Program. 

^^Take-up rates for programs without Five-Year Client Survey data were estimated from those with these data. 

Average statewide Medicaid costs per eligible month were calculated using 1995 data from the Health Care 
Financing Administration Web site (www.hcfa.gov). 

^^The costs of administering welfare, Food Stamp, and Medicaid benefits were estimated using statewide admin- 
istrative cost data. 

^"^One of the hypotheses of the case management experiment in Columbus was that integrated case management 
would allow more efficient delivery of services. As discussed in Scrivener and Walter, 2001, integrated case manag- 
ers reported seeing such benefits. This analysis assumes that the costs of administering benefits were the same for 
both programs. Because the five-year administration net costs were relatively low ($379 for the Integrated program 
and $265 for the Traditional program), even if the Integrated program had realized some savings in this area, the 
overall cost of the program would still have been slightly higher than the cost for the Traditional program. That is, 
the combined net operating and transfer administration costs for the Traditional program were $4,037, about $150 
less than the net operating cost of the Integrated program. 
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Table 13.8 
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The overall financial effect from the welfare sample perspective was estimated by sub- 
tracting the value of transfer payment losses from the value of gains in earnings, fringe benefits, 
and taxes. Clearly, programs where sample members experience large losses in transfer payments 
are less likely to produce net gains to sample members, because they have greater losses to offset 
with earned income. The three programs with the smallest welfare reductions (under $1,000 for 
both Atlanta programs and the Detroit program) produced very small net gains for sample mem- 
bers. However, these gains were very small; sample members effectively broke even. All of the 
programs with larger welfare reductions (ranging from about $1,148 in the Columbus Traditional 
program to $2,868 in the Riverside LFA program) produced net losses for sample members. 

As noted earlier, this analysis does not take into account losses of time for personal and 
family activities (leisure) and out-of-pocket expenditures associated with increases in employ- 
ment. Given that the three programs with gains did no more than break even, even small values 
attached to these effects would have resulted in losses for all programs. 

Table 13.9 shows the net present values from the perspective of welfare sample members 
who did not have a high school diploma or GED at the time they entered the evaluation. Only 
two programs (Detroit and Portland) produced gains for these welfare sample members. How- 
ever, these gains were small (sample members in Detroit effectively broke even). For the sub- 
group of sample members with a high school diploma or GED, three programs produced gains; 
both programs in Atlanta and the Detroit program. Again, gains were small. In fact, welfare sam- 
ple members in the Atlanta LFA and Detroit programs did little more than break even. Generally, 
those with a high school diploma did better (that is, bigger gains or smaller losses accrued to this 
subgroup). However, for the Grand Rapids LFA and Portland programs, this trend was reversed. 
Although welfare reductions in Portland were as large for nongraduates as for graduates, non- 
graduates experienced smaller losses in Food Stamps and Medicaid, resulting in smaller reduc- 
tions in total benefits from transfer programs. 

B. Government Budgets 

From the perspective of government budgets, programs are considered “budget neutral” if 
they can generate savings that equal the expenditures for services. It follows, then, that programs 
that produce large welfare savings have the best chance of paying for themselves and programs 
with small welfare savings will have a harder time breaking even. As shown in Table 13.10 the 
three programs (Grand Rapids LFA, Riverside LFA, and Portland) that resulted in gains to gov- 
ernment budgets had the largest welfare savings. The programs with the smallest welfare reduc- 
tions, ranging from $571 (Detroit) to $1,148 (Columbus Traditional), resulted in net losses. Pro- 
grams with welfare reductions in the midrange (Columbus Integrated and Grand Rapids HCD) 
essentially broke even. 

To provide some basis for interpreting these results and to facilitate cross-program com- 
parisons, Table 13.10 also presents an additional measure of the cost-effectiveness of the 
NEWWS programs from the government budget perspective. This measure is called the return to 
budget per net dollar invested, and it is calculated by dividing the gains (taxes and savings in 
transfer payments and associated administrative costs) by the total net costs of services. Using 
this metric, government budgets come Out ahead if programs produce more than a dollar’s worth 
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Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. 
Rounding may cause slight discrepancies in calculating sums and differences. 

Tests of statistical significance were not performed. 

^Payroll taxes include employer- and employee-paid Social Security and Medicare taxes. 

*^The return to budget per net dollar invested is computed by dividing total savings and tax increases by the net cost of activities and services. 



of additional revenues and savings for each dollar spent on employment-related services to pro- 
gram group members (compared with controls). 

Both the Portland and the Grand Rapids LFA programs produced over $2.00 in increased 
revenue and savings for every additional dollar spent on program group members. The Riverside 
LFA program also produced a considerable return, $1.47 per dollar invested. As noted above, the 
Grand Rapids HCD and Columbus Integrated programs essentially caused the government to 
break even ($0.92 to $1.06, respectively). The Atlanta, Columbus Traditional, and Detroit pro- 
grams were not as successful, returning considerably less than one dollar for each dollar invested, 
ranging from $0.41 in the Atlanta HCD program to just over $0.80 in the other programs. The 
average return across all of the programs was $1 .29 (not shown in any table). 

On average, these results are more positive than those found in the GAIN evaluation, in 
which returns to the government budget ranged from $0.17 per dollar invested (Tulare) to 
$2.84 (Riverside). The average across all counties in that study was $0.76. In the NEWWS 
Evaluation, the return on investments in Portland ($2.83) and Grand Rapids LFA ($2.46) com- 
pare favorably with the Riverside GAIN program and the SWIM program, which returned 
$2.84 and $2.34, respectively. 

From the perspective of government budgets, the results were mostly positive for those 
without a high school diploma or GED. As shown in Table 13.11, both Atlanta programs pro- 
duced losses. Gains ranged from $437 in Detroit to $5,564 in Grand Rapids LFA. For those with 
a high school diploma or GED, only the employment-focused programs paid for themselves (At- 
lanta LFA broke even). This is consistent with other findings: The welfare savings were larger 
for employment-focused programs; and those entering education-focused programs with a high 
school diploma or GED were more likely to participate in higher-cost activities, such as post- 
secondary education or vocational training, than those entering employment-focused (particularly 
LFA). However, this is not to say that programs with high rates of participation in higher-cost 
activities cannot pay for themselves: The mixed-approach Portland program had the highest re- 
turns to the government budget. 

XI. Sensitivity of the Results 

As is common in cost-benefit analyses, assumptions were made in producing the esti- 
mates in this analysis. Owing to data limitations, several major assumptions had to be made 
about the levels of participation in follow-up years 3 to 5, as well as the proportion of participa- 
tion that was considered in-program or out-of-program. The Five-Year Client Survey asked about 
participation at any point during the follow-up period, as well as specifically about participation 
in the last year of the follow-up period. This information was put together with data from the 
Two-Year Client Survey, which provided more detailed information about participation during 
the two-year period following program entry, to classify the likelihood of having participated 
during follow-up years 3 and 4. For the cost-benefit analysis, sample members were considered 
to have participated during years 3 and 4 if there was a good chance that they could have partici- 
pated during this period. 

In addition, the available data could not provide details on whether participation took 
place as part of the program or as a self-initiated activity. Therefore, iri the cost-benefit analysis. 
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Atlanta At lain ta Grand Rapids Grand Rapids Riverside Riverside Columbus Columbus 
Component of Analysis LFA ($) HCD ($) LFA ($) HCD ($) LFA ($) HCD ($) Integrated ($) Traditional ($) Detroit ($) Portland ($) 

With a high school 
diploma or GED 
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‘Payroll taxes include employer- and employee-paid Social Security and Medicare taxes. 

*^The return to budget per net dollar invested is computed by dividing total savings and tax increases by the net cost of activities and services. 



it was assumed that the proportion of in-program participation increased with the number of 
months of welfare receipt during the period in question. This was the best approximation of the 
participation patterns seen in the detailed two-year data used in earlier reports. 

To measure the effect of these two assumptions, a sensitivity test was conducted (see the 
following table). In this analysis, the alternative assumptions were that only those with the high- 
est likelihood of having participated in years 3 and 4 did so, and all participation was evenly di- 
vided between “in-program” and “out-of-program” for those who received welfare for a portion 
of the period. The resulting gross and net costs, as well as the net present value from the per- 
spective of goverrunent budgets, are presented below. (The welfare perspective is unaffected by 
program costs.) 



Site and Program 


Gross Cost 


Net Cost 
($) 


From the Government Budget Perspective 


Program 

Group 

($) 


Control 

Group 

($) 


Net Gain or 
Loss (NPV) 
($) 


Return to Budget per Net 
Dollar Invested in Program 
and Nonprogram Activitie.s 
and Services 

($) 


Atlanta LFA 


6,432 


3,006 


3,426 


-222 


0.94 


Atlanta HCD 


7,843 


3,006 


4,837 


-2,616 


0.46 


Grand Rapids LFA 


8,311 


6,513 


1,798 


3,097 


2.72 


Grand Rapids HCD 


9,993 


6,513 


3,480 


-15 


1.00 


Riverside LFA 


5,280 


2,538 


2,742 


2,123 


1.77 


Columbus Integrated 


6,158 


2,433 


3,724 


703 


1.19 


Columbus Traditional 


5,782 


2,433 


3,348 


-222 


0.93 


Detroit 


5,478 


3,751 


1,727 


-3 


1.00 


Portland 


8,456 


5,884 


2,572 


5,528 


3.15 . 



Using these alternative assumptions, gross costs were lower by 5 percent (Portland) to 13 
percent (Riverside LFA) for program group members and by 3 percent (Atlanta LFA and HCD 
and Portland) to 7 percent (Riverside LFA) for control group members. The alternative assump- 
tions resulted in reductions in net costs ranging from 8 percent (Grand Rapids HCD) to 17 per- 
cent (Riverside LFA). 

As these results show, the point estimates of gross and net costs are indeed sensitive to 
the assumptions made. However, using these lower cost estimates, the same conclusions were 
drawn with regard to the ranking of the various sites overall and between program approaches in 
the NEWWS Evaluation. In addition, the comparisons with the GAIN programs also remain 
largely unaffected. In the benefit-cost analysis, the lower cost estimates result in very few 



25t 



Both scenarios counted all participation as “in-program” for those who received welfare for the full period and 
all participation as “out-of-program” for those who did not receive any welfare during the entire period. 
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changes to the bottom line for government budgets: Although there were changes in the magni- 
tude of gains and losses, none of the estimates changed direction. In the sites where there were 
small losses to government budgets using the original assumptions, the losses are smaller yet, 
and these programs could be considered to have reached the “break-even” point. 

XII. Conclusions 

The costs of services were relatively high everywhere. This was true for controls as well. 
Gross costs for employment-focused programs were similar to those for education-focused pro- 
grams. However, net costs were lower for employment-focused programs. Similarly, LFA pro- 
grams continued to have lower costs than HCD programs. 

The NEWWS programs mostly left sample members worse off. Where there were gains 
to the welfare sample, they were very small. In half of the programs, the average earnings gains 
over the five-year period were smaller than reductions in welfare and Food Stamps. In Atlanta, 
which had the lowest welfare benefit levels, program group members had less to lose. As a result, 
both programs in this site produced earnings gains that were greater than the loss in combined 
welfare. Food Stamp, and Medicaid benefits. Detroit was the only other program to produce 
earnings gains greater than its reductions in transfer benefits. However, it bears repeating that the 
gains were very small and could easily have become losses had leisure and out-of-pocket work 
expenditures been included. 

Except for Portland and Grand Rapids LFA, sample members without a high school di- 
ploma or GED at random assignment fared worse than those with these credentials regardless of 
the program approach. However, even among those who entered the evaluation with a high 
school diploma or GED, programs mostly left sample members worse off. 

Although earnings increases were too small to cover benefit losses for program group 
members, in most programs the savings to government budgets were not large enough to cover 
their investments. The three programs that produced gains to government budgets were all em- 
ployment-focused and had among the lowest net costs. Similar results were seen for sample 
members who entered the programs with a high school diploma or GED. That is, within this sub- 
group, the four employment-focused programs broke even or produced gains to government 
budgets, and all of the education-focused programs produced losses. For the subgroup without 
these credentials, all programs, except for the two in Atlanta, produced gains to government 
budgets. In general, the greater investment of education-focused programs did not pay off for 
government budgets or for program group members. 

For the full samples, no program produced gains to both the welfare sample and govern- 
ment budgets. This was true for the educational attainment subgroups as well, except for non- 
graduates in Portland: Small gains ($1,084 over five years) accrued to nongraduates in that pro- 
gram, and the government budget experienced a moderate gain of $2,787. (Detroit produced a 
small gain to the government budget but broke even from the perspective of the welfare sample.) 
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Appendix A 



Notes for Tables and Figures 



Appendix A.1 



Notes for Tables and Figures Displaying Impacts 
Calculated with Administrative Records Data 



Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 
assignment characteristics of sample members. 

“Percentage change” equals 100 times “difference” divided by “control group.” 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. Statistical significance levels are indicated as: * = 10 percent; 

** = 5 percent; and *** = 1 percent. 

Riverside limited enrollment in its HCD program to individuals determined by program 
regulations to need basic education because they lacked a high school diploma or GED 
certificate, attained low scores on a reading or math test administered at program entry, or had 
limited proficiency in English. As a result, control group means differ for the Riverside LFA and 
HCD programs. 

If outcomes are shown in italics, differences between program group members and control group 
members are not true experimental comparisons; statistical tests were not performed. 

The quarter of random assignment refers to the calendar quarter in which random assignment 
occurred. Because quarter 1, the quarter of random assignment, may contain some earnings from 
the period prior to random assignment, it is excluded from follow-up measures. Thus, “year 1” is 
quarters 2 through 5, “year 2” is quarters 6 through 9, “year 3” is quarters 10 through 13, “year 4” 
is quarters 14 through 17, and “year 5” is quarters 18 through 21. 

N/A = not applicable. 
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Appendix A.2 



Notes for Tables and Figures Displaying Impacts Calculated with Responses to the Two- 
Year Client Survey, Five-Year Client Survey, Child Outcomes Study, 

and Teacher Survey 



Measures for program and control group members represent weighted averages. In all sites, 
certain subgroups were overrepresented (for research purposes) among those chosen to be 
surveyed. Members of the survey samples are weighted to replicate the proportion of program 
and control group members in the full impact sample. 

The Five-Year Client Survey sample includes 434 respondents who were not interviewed for the 
Two-Year Client Survey. Measures calculated from responses to both surveys exclude these 
sample members. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 
assignment characteristics of sample members. 

“Percentage change” equals 100 times “difference” divided by “control group.” 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. Statistical significance levels are indicated as: * = 10 percent; 

** = 5 percent; and *** = i percent. 

Riverside limited enrollment in its HCD program to individuals determined by program 
regulations to, need basic education because they lacked a high school diploma or GED 
certificate, attained low scores on a reading or math test administered at program entry, or had 
limited proficiency in English. As a result, control group means differ for the Riverside LFA and 
HCD programs. 

If outcomes are shown in italics, differences between program group members and control group 
members are not true experimental comparisons; statistical tests were not performed. 

In Chapters 1 1 and 12, “effect size” equals “difference” divided by the standard deviation (not 
shown in tables) in the “control group.” If measures are presented as percentages, “effect size” 
equals “difference” divided by 100, then divided by the standard deviation in the “control group.” 
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Appendix B 



Supplementary Tables to Chapter 3 



National Evaluation of Welfare-to-Work Strategies 
Appendix Table B.l 



Five-Year Impacts on Participation in Employment-Related Activities 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 








Basic education 




Atlanta Labor Force Attachment 


1,071 


20.4 


11.4 


9 Y ♦♦♦ 


79.8 


Atlanta Human Capital Development 


1,146 


29.9 


11.4 


18.5 ♦♦♦ 


163.0 


Grand Rapids Labor Force Attachment 


1,097 


20.0 


18.9 


1.2 


6.3 


Grand Rapids Human Capital Development 


1,109 


32.1 


18.9 


13.2 ♦♦♦ 


70.2 


Riverside Labor Force Attachment 


1,219 


17.1 


17.9 


-0.8 


-4.5 


Lacked high school diploma or basic skills 


657 


24.9 


28.0 


-3.2 


-11.3 


Riverside Human Capital Development 


778 


59.0 


28.0 


31.0 ♦♦♦ 


110.6 


Portland 


504 


27.2 


20.9 


6.3 ♦ 


30.2 






Post-secondary education 




Atlanta Labor Force Attachment 


1,071 


7.5 


8.7 


-1.2 


-13.5 


Atlanta Human Capital Development 


1,146 


11.5 


8.7 


2.8 


31.7 


Grand Rapids Labor Force Attachment 


1,097 


27.0 


26.2 


0.7 


2.8 


Grand Rapids Human Capital Development 


1,109 


29.6 


26.2 


3.3 


12.7 


Riverside Labor Force Attachment 


1,219 


24.5 


25.0 


-0.5 


-1.9 


Lacked high school diploma or basic skills 


657 


12.6 


15.5 


-2.9 


-18.7 


Riverside Human Capital Development 


778 


13.1 


15.5 


-2.4 


-15.2 


Portland 


504 


42.4 


26.0 


16.4 ♦♦♦ 


63.0 








Vocational training 




Atlanta Labor Force Attachment 


1,071 


23.8 


23.2 


0.6 


2.7 


Atlanta Human Capital Development 


1,146 


33.0 


23.2 


9 y ♦♦♦ 


42.0 


Grand Rapids Labor Force Attachment 


1,097 


23.3 


24.2 


-0.8 


-3.5 


Grand Rapids Human Capital Development 


1,109 


28.5 


24.2 


4.3 


17.7 


Riverside Labor Force Attachment 


1,219 


23.7 


24.5 


-0.8 


-3.4 


Lacked high school diploma or basic skills 


657 


21.5 


20.9 


0.6 


2.8 


Riverside Human Capital Development 


778 


21.2 


20.9 


0.3 


1.4 


Portland 


504 


25.3 


28.7 


-3.4 


-11.8 



(continued) 
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Appendix Table B.l (continued) 



Site and Program 


Sample 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 








Work experience 




Atlanta Labor Force Attachment 


1,071 


17.2 


7.6 


9.6 *** 


126.3 


Atlanta Human Capital Development 


1,146 


12.2 


7.6 


4.6 ** 


61.2 


Grand Rapids Labor Force Attachment 


1,097 


10.5 


5.2 


5.3 *** 


101.1 


Grand Rapids Human Capital Development 


1,109 


7.6 


5.2 


2.4 


45.3 


Riverside Labor Force Attachment 


1,219 


7.8 


3.2 


4.6 *** 


147.1 


Lacked high school diploma or basic skills 


657 


7.3 


2.2 


5.2 *** 


235.9 


Riverside Human Capital Development 


778 


6.3 


2.2 


4.2 ** 


190.0 


Portland 


504 


14.7 


13.7 


1.0 


7.1 



SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A.2. 



National Evaluation of Welfare-to-Work Strategies 
Appendix Table B.2 

Five-Year Impacts on Participation in Employment-Related Activities 
for Sample Members Without a High School Diploma or GED at Random Assignment 



Site and Program 


Sample Program Control Difference 

Size Group (%) Group (%) (Impact) 


Percentage 
Change (%) 






Basic education 




Atlanta Labor Force Attachment 


407 


34.5 


18.5 


16.0 *** 


86.6 


Atlanta Human Capital Development 


437 


59.8 


18.5 


41.3 *** 


223.1 


Grand Rapids Labor Force Attachment 


434 


44.8 


41.8 


3.0 


7.2 


Grand Rapids Human Capital Development 


468 


68.8 


41.8 


27 Q 


64.8 


Riverside Labor Force Attachment 


657 


24.9 


28.0 


-3.2 


-11.3 


Riverside Human Capital Development 


778 


59.0 


28.0 


31.0 *** 


110.6 


Portland 


163 


59.3 


49.5 


9.8 


19.7 






Post-secondary education 




Atlanta Labor Force Attachment 


407 


1.2 


0.8 


0.3 


37.1 


Atlanta Human Capital Development 


437 


2.5 


0.8 


1.6 


193.9 


Grand Rapids Labor Force Attachment 


434 


9.3 


6.4 


2.9 


45.8 


Grand Rapids Human Capital Development 


468 


11.1 


6.4 


4.8 * 


74.5 


Riverside Labor Force Attachment 


657 


12.6 


15.5 


-2.9 


-18.7 


Riverside Human Capital Development 


778 


13.1 


15.5 


-2.4 


-15.2 


Portland 


163 


21.6 


13.8 


7.8 


56.9 






Vocational training 




Atlanta Labor Force Attachment 


407 


14.0 


16.3 


-2.3 


-14.3 


Atlanta Human Capital Development 


437 


15.2 


16.3 


-1.1 


-6.8 


Grand Rapids Labor Force Attachment 


434 


19.9 


18.9 


1.0 


5.2 


Grand Rapids Human Capital Development 


468 


20.6 


18.9 


1.7 


9.0 


Riverside Labor Force Attachment 


657 


21.5 


20.9 


0.6 


2.8 


Riverside Human Capital Development 


778 


21.2 


20.9 


0.3 


1.4 


Portland 


163 


20.4 


29.6 


-9.2 


-31.2 



(continued) 
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Appendix Table B.2 (continued) 



Site and Program 


Sample 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 








Work experience 




Atlanta Labor Force Attachment 


407 


14.5 


7.2 


7 3 ♦♦ 


101.2 


Atlanta Human Capital Development 


437 


8.4 


7.2 


1.2 


16.1 


Grand Rapids Labor Force Attachment 


434 


9.7 


5.5 


4.1 ♦ 


74.9 


Grand Rapids Human Capital Development 


468 


4.7 


5.5 


-0.8 


-15.3 


Riverside Labor Force Attachment 


657 


7.3 


2.2 


^ 2 ♦♦♦ 


235.9 


Riverside Human Capital Development 


778 


6.3 


2.2 


4.2 ♦♦ 


190.0 


Portland 


163 


17.8 


7.9 


9.9 


125.3 



SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A.2. 
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National Evaluation of Welfare-to-Work Strategies 
Appendix Table B.3 

Five-Year Impacts on Participation in Employment-Related Activities 
for Sample Members With a High School Diploma or GED at Random Assignment 

Sample Program Control Difference Percentage 

Site and Program Size Group (%) Group (%) (Impact) Change (%) 

Basic education 



Atlanta Labor Force Attachment 664 

Atlanta Human Capital Development 709 

Grand Rapids Labor Force Attachment 663 

Grand Rapids Human Capital Development 641 

Riverside Labor Force Attachment 562 

Portland 334 

Atlanta Labor Force Attachment 664 

Atlanta Human Capital Development 709 

Grand Rapids Labor Force Attachment 663 

Grand Rapids Human Capital Development 641 

Riverside Labor Force Attachment 562 

Portland 334 

Atlanta Labor Force Attachment 664 

Atlanta Human Capital Development 709 

Grand Rapids Labor Force Attachment 663 

Grand Rapids Human Capital Development 641 

Riverside Labor Force Attachment 562 

Portland 334 



11.4 


6.9 


4.5 ** 


65.1 


11.1 


6.9 


4.1 * 


59.8 


6.3 


6.6 


-0.2 


-3.4 


12.9 


6.6 


6.4 *** 


97.3 


6.7 


4.7 


2.0 


42.7 


12.2 


6.8 


5.4 


79.7 



Post-secondary education 



11.5 


13.7 


-2.2 


-16.1 


16.8 


13.7 


3.1 


22.7 


36.7 


37.1 


-0.4 


-1.1 


39.4 


37.1 


2.3 


6.1 


40.5 


37.7 


2.8 


7.6 


52.6 


31.7 


20.9 *** 


66.2 



Vocational training 



30.7 


27.1 


3.7 


13.6 


43.7 


27.1 


16.6 *** 


61.5 


25.4 


■ 27.1 


-1.6 


-6.1 


32.5 


27.1 


5.4 


20.0 


26.6 


29.2 


-2.6 


-8.8 


27.7 


29.1 


-1.4 


-4.8 



(continued) 



Appendix Table B.3 (continued) 



Site and Program 


Sample 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 








Work experience 




Atlanta Labor Force Attachment 


664 


18.7 


7.8 


10.9 *** 


140.7 


Atlanta Human Capital Development 


709 


14.8 


7.8 


7.0 *** 


90.2 


Grand Rapids Labor Force Attachment 


663 


11.0 


4.9 


6.1 *** 


124.3 


Grand Rapids Human Capital Development 


641 


9.2 


4.9 


4.3 ** 


88.2 


Riverside Labor Force Attachment 


562 


8.4 


4.4 


4.0 * 


91.0 


Portland 


334 


12.9 


15.7 


-2.8 


-17.7 



SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A.2. 
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Appendix Table C.l 

Impacts on Employment in Years 1 to 5 



Site and Program 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Atlanta Labor Force Attachment 










Year 1 


49.1 


44.4 


4.7 *** 


10.6 


Year 2 


55.3 


49.8 


5.5 *** 


11.0 


Year 3 


58.4 


54.1 


4.4 


8.1 


Year 4 


61.5 


58.5 


3.0 * 


5.2 


Year 5 


65.1 


63.0 


2.1 


3.3 


Sample size 


1,441 


1,497 






Atlanta Human Capital Development 










Year 1 


'45.5 


44.4 


1.1 


2.5 


Year 2 


54.3 


49.8 


4.5 *** 


9.0 


Year 3 


58.2 


54.1 


4.2 ** 


7.7 


Year 4 


60.3 


58.5 


1.8 


3.1 


Years 


63.6 


63.0 


0.6 


1.0 


Sample size 


1,495 


1,497 






Grand Rapids Labor Force Attachment 










Year 1 


63.6 


52.9 


10.7 *** 


20.2 


Year 2 


67.2 


60.6 


6.6 *** 


10.8 


Year 3 


69.8 


65.0 


4.7 *** 


7-3 


Year 4 


71.3 


70.0 


1.3 


1.9 


Year 5 


70.0 


73.0 


-2.9 * 


-4.0 


Sample size 


1,557 


1,455 






Grand Rapids Human Capital Development 










Year 1 


58.1 


52.9 


5.3 *** 


9.9 


Year 2 


65.5 


60.6 


4.9 *** 


8.0 


Year 3 


67.2 


65.0 


2.2 


3.4 


Year 4 


70.0 


70.0 


0.0 


0.0 


Years 


70.3 


73.0 


-2.7 * 


-3.7 


Sample size 


1,542 


1,455 






Riverside Labor Force Attachment 










Year 1 


50.6 


34.1 


16.5 *** 


48.5 


Year 2 


45.4 


37.0 


8.4 *** 


22.6 


Year 3 


44.5 


39.8 


4.7 *** 


11.9 


Year 4 


45.8 


41.9 


3.9 *** 


9.2 


Years 


48.7 


44.5 


4.2 *** 


9.5 


Sample size 


3,384 


3,342 







(continued) 
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Appendix Table C.l (continued) 



Program Control Difference Percentage 

Site and Program Group (%) Group (%) (Impact) Change (%) 



Riverside Labor Force Attachment 

Lacked high school diploma or basic skills 

Year 1 

Year 2 

Year 3 

Year 4 

Year 5 

Sample size 

Riverside Human Capital Development 

Year 1 

Year 2 

Year 3 

Year 4 

Year 5 

Sample size 

Columbus Integrated 

Year 1 

Year 2 

Year 3 

Year 4 

Years 

Sample size 

Columbus Traditional 

Year 1 

Year 2 

Year 3 

Year 4 

Year 5 

Sample size 

Detroit 
Year 1 
Year 2 
Year 3 
Year 4 
Year 5 

Sample size 



45.8 


29.1 


16.7 *** 


57.4 


40.9 


31.9 


9.0 *** 


28.3 


40.8 


34.8 


6.1 *** 


17.5 


41.5 


37.1 


4.4 *** 


11.9 


45.8 


39.9 


5.9 *** 


14.8 


1,586 


1,539 







35.7 


29.1 


6.6 *** 


22.9 


37.7 


31.9 


5.8 *** 


18.2 


39.3 


34.8 


4.6 *** 


13.2 


40.9 


37.1 


3.9 ** 


10.4 


44.9 


39.9 


5.0 *** 


12.5 


1,596 


1,539 







60.0 


60.1 


-0.1 


-0.2 


65.2 


62.9 


2.3 * 


3.7 


68.9 


65.3 


3.6 *** 


5.5 


69.9 


67.7 


2.2 * 


3.2 


69.1 


68.8 


0.3 


0.5 


2,513 


2,159 







59.9 


60.1 


-0.1 


-0.2 


64.5 


62.9 


1.6 


2.6 



67.9 65.3 2.6 ** 3.9 



68.9 


67.7 


1.2 


1.8 


69.3 


68.8 


0.5 


• 0.7 


2,570 


2,159 







41.2 


40.6 


0.6 


1.6 


54.2 


51.5 


2.6 * 


5.1 


62.3 


59.0 


3.3 ** 


5.5 


66.4 


67.5 


-1.1 


-1.7 


68.8 


68.8 


0.0 


-0.1 


2,226 


2,233 







(continued) 





Appendix Table C.l (continued) 



Site and Program 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Oklahoma City 










Year 1 


50.9 


51.6 


-0.7 


-1.4 


Year 2 


51.0 


51.6 


-0.6 


-1.2 


Years 


51.4 


52.1 


-0.7 


-1.4 


Year 4 


52.9 


52.7 


0.3 


0.5 


Year 5 


53.2 


54.2 


-1.0 


-1.8 


Sample size 


4,309 


4,368 






Portland 










Year 1 


58.5 


49.8 


8.6 ••• 


17.3 


Year 2 


62.3 


51.4 


10.9 ••• 


21.2 


Years 


64.1 


52.0 


12.1 ••• 


23.2 


Year 4 


64.2 


55.8 


8.5 ••• 


15.2 


Year 5 


62.4 


58.6 


bo 

« 


6.4 


Sample size 


3,529 


499 







SOURCE: MDRC calculations from state and county administrative records. 



NOTES: See Appendix A. 1 . 
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Appendix Table C.2 

Impacts on Earnings in Years 1 to 5 



Site and Program 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Atlanta Labor Force Attachment 










Year 1 


2,043 


1,647 


396 *** 


24.0 


Year 2 


3,228 


2,578 


650 *** 


25.2 


Years 


3,930 


3,313 


618 *** 


18.7 


Year 4 


4,815 


4,255 


560 ** 


^ 13.2 


Year 5 


5,821 


5,586 


235 


4.2 


Sample size 


1,441 


1,497 






Atlanta Human Capital Development 










Year 1 


1,846 


1,647 


199 * 


12.1 


Year 2 


3,059 


2,578 


480 *** 


18.6 


Years 


3,912 


3,313 


*** 


18.1 


Year 4 


4,910 


4,255 


655 *** 


15.4 


Years 


5,671 


5,586 


84 


1.5 


Sample size 


1,495 


1,497 






Grand Rapids Labor Force Attachment 










Year 1 


2,289 


1,759 


530 *** 


30.2 


Year 2 


3,385 


2,874 


♦♦♦ 


17.8 


Years 


4,658 


4,211 


447 ** 


10.6 


Year 4 


5,615 


5,480 


135 


2.5 


Years 


6,376 


6,447 


-71 


-1.1 


Sample size 


1,557 


1,455 






Grand Rapids Human Capital Development 










Year 1 


1,869 


1,759 


no 


6.3 


Year 2 


3,354 


2,874 


479 ♦♦♦ 


16.7 


Year S 


4,514 


4,211 


303 


7.2 


Year 4 


5,535 


5,480 


56 


1.0 


Years 


6,345 


6,447 


-102 


-1.6 


Sample size 


1,542 


1,455 






Riverside Labor Force Attachment 










Year 1 


2,460 


1,741 


7J9 ♦♦♦ 


41.3 


Year 2 


3,039 


2,482 


557 ♦♦♦ 


22.4 


Years 


3,389 


2,957 


432 *** 


14.6 


Year 4 


3,867 


3,554 


313 * 


8.8 


Years 


4,683 


4,155 


528 *** 


12.7 


Sample size 


3,384 


3,342 







(continued) 
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Appendix Table C.2 (continued) 



Program Control Difference Percentage 

Site and Program Group (%) Group (%) (Impact) Change (%) 



Riverside Labor Force Attachment 

Lacked high school diploma or basic skills 

Year 1 

Year 2 

Year 3 

Year 4 

Years 

Sample size 

Riverside Human Capital Development 

Year 1 

Year 2 

Year 3 

Year 4 

Years 

Sample size 

Columbus Integrated 

Year 1 

Year 2 

Year 3 

Year 4 

Year S 

Sample size 

Columbus Traditional 

Year 1 

Year 2 

Year 3 

Year 4 

Years 

Sample size 

Detroit 
Year 1 
Year 2 
Year 3 
Year 4 
Years 

Sample size 



1,866 


1,250 


616 *** 


49.3 


2,270 


1,886 


384 ** 


20.4 


2,541 


2,150 


391 ** 


18.2 


2,882 


2,601 


281 


10.8 


3,634 

1,586 


3,025 

1,539 


608 *** 


20.1 



1,446 


1,250 


196 * 


15.7 


2,025 


1,886 


139 


7.4 


2,561 


2,150 


411 ** 


19.1 


2,857 


2,601 


255 


9.8 


3,384 

1,596 


3,025 

1,539 


359 


11.9 



2,994 


2,914 


80 


2.8 


4,578 


3,982 




15.0 


5,644 


5,134 


510 *** 


9.9 


6,722 


6,260 


463 ** 


7.4 


7,682 

2,513 


7,276 

2,159 


406 


5.6 



3,099 


2,914 


185 


6.4 


4,472 


3,982 


490 *** 


12.3 


5,459 


5,134 


325 * 


6.3 


6,557 


6,260 


298 


4.8 


7,389 

2,570 


7,276 

2,159 


113 


1.6 



1,398 


1,341 


57 


4.2 


2,976 


2,663 


313 ** 


11.8 


4,593 


4,112 


481 ** 


11.7 


5,920 


5,570 


351 


6.3 


7,081 

2,226 


6,823 

2,233 


258 


3.8 



(continued) 




-3S6- 

30S 



Appendix Table C.2 (continued) 





Program 


Control 


Difference 


Percentage 


Site and Program 


Group (%) 


Group (%) 


(Impact) 


Change (%) 



Oklahoma City 



Year 1 


1,404 


1,387 


16 


1.2 


Year 2 


2,121 


2,127 


-5 


-0.3 


Year 3 


2,632 


2,672 


-40 


-1.5 


Year 4 


3,114 


3,068 


45 


1.5 


Year 5 
Sample size 


3,597 

4,309 


3,498 

4,368 


100 


2.8 


Portland 


Year 1 


2,828 


2,484 


343 * 


13.8 


Year 2 


4,421 


3,150 


1,271 *** 


40.4 


Year 3 


5,491 


4,050 


1,441 *** 


35.6 


Year 4 


6,319 


5,112 


1,207 *** 


23.6 


Year 5 
Sample size 


6,982 

3,529 


6,095 

499 


887 ** 


14.6 



SOURCE: MDRC calculations from state and county administrative records. 



NOTES: See Appendix A. 1 . 
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Appendix Table C.3 

Impacts on Employment and Earnings in Years 1 to 3 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Inmact) 


Percentage 
Change (%) 








Ever emoloved (Vo) 




Atlanta Labor Force Attachment 


2,938 


72.0 


68.2 


3.8 ** 


5.6 


Atlanta Human Capital Development 


2,992 


71.3 


68.2 


3.0 * 


4.4 


Grand Rapids Labor Force Attachment 


3,012 


84.0 


77.6 


5 4 


8.2 


Grand Rapids Human Capital Development 


2,997 


82.0 


77.6 


4 4 


5.6 


Riverside Labor Force Attachment 


6,726 


66.1 


53.8 


12.3 *** 


22.8 


Lacked high school diploma or basic skills 


3,125 


61.5 


47.1 


14 3 *** 


30.4 


Riverside Human Capital Development 


3,135 


55.9 


47.1 


g g *** 


18.7 


Columbus Integrated 


4,672 


81.1 


78.5 


2.6 ** 


3.3 


Columbus Traditional 


4,729 


80.7 


78.5 


2 2 ** 


2.8 


Detroit 


4,459 


74.0 


70.4 


3 6 *** 


5.1 


Oklahoma City 


8,677 


71.5 


71.8 


- 0.4 


- 0.5 


Portland 


4,028 


80.3 


71.5 


g g *** 


12.2 




Average number of Quarters emoloyed (%) 


Atlanta Labor Force Attachment 


2,938 


4.6 


4.0 


0.6 *** 


14.0 


Atlanta Human Capital Development 


2,992 


4.4 


4.0 


Q 4 *** 


9.7 


Grand Rapids Labor Force Attachment 


3,012 


5.4 


4.7 


Q y 


15.7 


Grand Rapids Human Capital Development 


2,997 


5.1 


4.7 


Q 4 *** 


8.5 


Riverside Labor Force Attachment 


6,726 


3.9 


3.0 


Q 9 414141 


28.9 


Lacked high school diploma or basic skills 


3,125 


3.4 


2.5 


Q Q 


36.2 


Riverside Human Capital Development 


3,135 


2.9 


2.5 


0 5 *** 


18.2 


Columbus Integrated 


4,672 


5.8 


5.5 


Q 3 


5.3 


Columbus Traditional 


4,729 


5.7 


5.5 


0.2 ** 


4.1 


Detroit 


4,459 


4.1 


3.9 


0.2 ** 


6.1 


Oklahoma City 


8,677 


3.7 


3.8 


- 0.1 


- 2.9 


Portland 


4,028 


5.4 


4.2 


j j *** 


26.3 
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Appendix Table C.3 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Percentage 
Change (%) 






Avera 2 e total earninss ($) 




Atlanta Labor Force Attachment 


2,938 


9,202 


7,538 


1,664 *** 


22.1 


Atlanta Human Capital Development 


2,992 


8,816 


7,538 


1,278 *** 


17.0 


Grand Rapids Labor Force Attachment 


3,012 


10,332 


8,844 


1,488 *** 


16.8 


Grand Rapids Human Capital Development 


2,997 


9,736 


8,844 


892 ** 


10.1 


Riverside Labor Force Attachment 


6,726 


8,888 


7,180 


1,708 *** 


23.8 


Lacked high school diploma or basic skills 


3,125 


6,678 


5,286 


1,392 *** 


26.3 


Riverside Human Capital Development 


3,135 


6,032 


5,286 


746 ** 


14.1 


Columbus Integrated 


4,672 


13,216 


12,030 


■ 1,186 *** 


9.9 


Columbus Traditional 


4,729 


13,030 


12,030 


1,000 ** 


8.3 


Detroit 


4,459 


8,967 


8,115 


851 ** 


10.5 


Oklahoma City 


8,677 


6,157 


6,186 


-29 


-0.5 


Portland 


4,028 


12,740 


9,684 


3,055 *** 


31.5 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1. 
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Appendix Table C.4 

Impacts on Employment and Earnings in the Last Quarter of Year 5 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Inmact) 


Percentage 
Change (%) 








Ever emoloved 




Atlanta Labor Force Attachment 


2,938 


50.9 


51.5 


-0.5 


-1.1 


Atlanta Human Capital Development 


2,992 


51.3 


51.5 


-0.2 


-0.4 


Grand Rapids Labor Force Attachment 


3,012 


54.9 


■59.0 


-4.1 ♦♦ 


-6.9 


Grand Rapids Human Capital Development 


2,997 


56.7 


59.0 


-2.3 


-4.0 


Riverside Labor Force Attachment 


6,726 


38.4 


35.5 


2.9 ♦♦ 


8.1 


Lacked high school diploma or basic skills 


3,125 


35.3 


31.1 


4.2 ♦♦ 


13.5 


Riverside Human Capital Development 


3,135 


33.8 


31.1 


2.7 ♦ 


8.8 


Columbus Integrated 


4,672 


54.4 


55.0 


-0.6 


-1.2 


Columbus Traditional 


4,729 


54.2 


55.0 


-0.8 


-1.5 


Detroit 


4,459 


55.0 


55.2 


-0.2 


-0.4 


Oklahoma City 


8,677 


37.3 


38.2 


-0.9 


-2.4 


Portland 


4,028 


50.1 


44.7 


5.4 ♦♦ 


12.2 








Total earnings TS) 




Atlanta Labor Force Attachment 


2,938 


1,544 


1,496 


■ 48 


3.2 


Atlanta Human Capital Development 


2,992 


1,469 


1,496 


-26 


-1.7 


Grand Rapids Labor Force Attachment 


3,012 


1,659 


1,746 


-87 


-5.0 


Grand Rapids Human Capital Development 


2,997 


1,675 


1,746 


-71 


-4.1 


Riverside Labor Force Attachment 


6,726 


1,236 


1,113 


123 ** 


11.0 


Lacked high school diploma or basic skills 


3,125 


986 


829 


156 ** 


18.9 


Riverside Human Capital Development 


3,135 


885 


829 


56 


6.8 


Columbus Integrated 


4,672 


2,008 


1,914 


94 


4.9 


Columbus Traditional 


4,729 


1,913 


1,914 


-1 


0.0 


Detroit 


4,459 


1,847 


1,849 


-2 


-0.1 


Oklahoma City 


8,677 


934 


939 


-5 


-0.5 


Portland 


4,028 


1,793 


1,555 


238 ** 


15.3 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES : See Appendix A. 1 . 
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Appendix Table C.5 
Impacts on Employment 

Sample Program Control Difference Percentage 

Site and Program Size Group (%) Group (%) (Impact) Change (%) 



Atlanta Labor Force Attachment 1,071 

Atlanta Human Capital Development 1,146 

Grand Rapids Labor Force Attachment 1,097 

Grand Rapids Human Capital Development 1 , 1 09 

Riverside Labor Force Attachment 1,219 

Lacked high school diploma or basic skills 657 

Riverside Human Capital Development 778 

Portland 504 

Atlanta Labor Force Attachment 1,071 

Atlanta Human Capital Development 1,146 

Grand Rapids Labor Force Attachment 1,097 

Grand Rapids Human Capital Development 1 , 1 09 

Riverside Labor Force Attachment 1,219 

Lacked high school diploma or basic skills 657 

Riverside Human Capital Development 778 

Portland 504 

Atlanta Labor Force Attachment 1,071 

Atlanta Human Capital Development 1,146 

Grand Rapids Labor Force Attachment 1 ,097 

Grand Rapids Human Capital Development 1 , 1 09 

Riverside Labor Force Attachment 1,219 

Lacked high school diploma or basic skills 657 

Riverside Human Capital Development 778 

Portland 504 



Ever employed in years 1 to 5 




83.3 


82.9 


0.4 


0.4 


81.2 


82.9 


-1.7 


-2.0 


94.7 


91.9 


2.9 * 


3.1 


93.6 


91.9 


1.7 


1.9 


85.7 


80.8 


4 ^ 


6.0 


81.8 


74.2 


7.6 ** 


10.2 


82.8 


74.2 


8.6 *** 


11.6 


93.5 


93.3 


0.2 


0.2 


Employed at interview 





57.2 


54.3 


2.9 


5.4 


52.9 


54.3 


-1-4 


-2.5 


65.1 


67.7 


-2.6 


-3.8 


66.2 


67.7 


-1.5 


-2.2 


55.0 


48.9 


6.1 ** 


12.4 


48.9 


43.0 


5.9 


13.6 


51.1 


43.0 


8.0 ** 


18.7 


61.7 


58.3 


3.4 


5.9 



Employed full time at interview 



47.2 


45.5 


1.7 


3.8 


44.1 


45.5 


-1.4 


-3.2 


53.5 


54.3 


-0.8 


-1.5 


52.9 


54.3 


-1.3 


-2.5 


44.1 


35.9 


g 2 ♦♦♦ 


22.7 


40.2 


31.8 


8.4 ** 


26.4 


36.8 


31.8 


4.9 


15.4 


50.3 


46.9 


3.4 


7.2 



(continued) 



Appendix Table C.5 (continued) 



Sample Program Control Difference Percentage 

Site and Program Size Group (%) Group (%) (Impact) Change (%) 

Employed with employer-provided medical 
insurance at interview 



Atlanta Labor Force Attachment 1 ,07 1 

Atlanta Human Capital Development 1,146 

Grand Rapids Labor Force Attachment 1 ,097 

Grand Rapids Human Capital Development 1 , 1 09 

Riverside Labor Force Attachment 1,219 

Lacked high school diploma or basic skills 657 

Riverside Human Capital Development 778 

Portland 504 



22.2 


19.0 


3.2 


16.6 


21.2 


19.0 


2.2 


11.5 


30.5 


30.0 


0.5 


1.7 


30.4 


30.0 


0.4 


1.5 


17.5 


15.0 


2.5 


16.6 


12.0 


10.9 


1-1 


9.8 


14.8 


10.9 


3.9 


35.5 


31.6 


27.3 


4.4 


16.0 



SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A.2. 
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Appendix Table C.6 



Impacts on Longer-Term Employment Stability and Earnings Growth 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 






Emnloved in 75 oercent or more of 








Quarters in vears 3 to 5 




Atlanta Labor Force Attachment 


2,938 


37.0 


32.5 


4.5 ♦♦♦ 


14.0 


Atlanta Human Capital Development 


2,992 


35.6 


32.5 


3.1 ♦ 


9.5 


Grand Rapids Labor Force Attachment 


3,012 


40.8 


38.0 


2.9 ♦ 


7.6 


Grand Rapids Human Capital Development 


2,997 


39.8 


38.0 


1.8 


4.8 


Riverside Labor Force Attachment 


6,726 


23.7 


20.6 


3.2 ♦♦♦ 


15.5 


Lacked high school diploma or basic skills 


3,125 


20.0 


16.2 


3.8 *** 


23.4 


Riverside Human Capital Development 


3,135 


20.1 


16.2 


3 9 ♦♦♦ 


24.0 


Columbus Integrated 


4,672 


43.7 


42.4 


1.3 


3.1 


Columbus Traditional 


4,729 


43.4 


42.4 


1.0 


2.2 


Detroit 


4,459 


35.9 


34.3 


1.7 


4.9 


Oklahoma City 


8,677 


22.1 


22.8 


-0.6 


-2.7 


Portland 


4,028 


38.6 


31.2 


7.5 *** 


23.9 






Increased average earnings from first to last 






measured Quarters in vears 1 to 5 




Atlanta Labor Force Attachment 


2,938 


40.4 


38.6 


1.8 


4.7 


Atlanta Human Capital Development 


2,992 


40.8 


38.6 


2.3 


5.9 


Grand Rapids Labor Force Attachment 


3,012 


46.8 


46.0 


0.8 


1.6 


Grand Rapids Human Capital Development 


2,997 


46.3 


46.0 


0.3 


0.7 


Riverside Labor Force Attachment 


6,726 


31.9 


27.4 


4 4 ♦♦♦ 


16.2 


Lacked high school diploma or basic skills 


3,125 


28.2 


21.8 


6.4 


29.4 


Riverside Human Capital Development 


3,135 


26.9 


21.8 


5.1 *** 


23.2 


Columbus Integrated 


4,672 


48.9 


46.6 


2.3 


4.9 


Columbus Traditional 


4,729 


48.2 


46.6 


1.6 


3.5 


Detroit 


4,459 


42.0 


42.3 


-0.3 


-0.8 


Oklahoma City 


8,677 


30.4 


30.0 


0.4 


1.2 


Portland 


4,028 


45.2 


38.3 


6.9 *** 


17.9 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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National Evaluation of Welfare-to-Work Strategies 
Appendix Table D.l 

Impacts on Welfare Receipt and Payments in Years 1 to 3 



Sample Program Control Difference Percentage 

Site and Program Size Group Group (Impact) Change (%) 

Average number of months receiving welfare 
in years 1 to 3 



Atlanta Labor Force Attachment 2,938 

Atlanta Human Capital Development 2,992 

Grand Rapids Labor Force Attachment 3,01 2 

Grand Rapids Human Capital Development 2,997 

Riverside Labor Force Attachment 6,726 

Lacked high school diploma or basic skills 3,125 

Riverside Human Capital Development 3,135 

Columbus Integrated 4,672 

Columbus Traditional 4,729 

Detroit 4,459 

Oklahoma City 6,896 

Portland 4,028 

Atlanta Labor Force Attachment 2,938 

Atlanta Human Capital Development 2,992 

Grand Rapids Labor Force Attachment 3,0 1 2 

Grand Rapids Human Capital Development 2,997 

Riverside Labor Force Attachment 6,726 

Lacked high school diploma or basic skills 3,125 

Riverside Human Capital Development 3,135 

Columbus Integrated 4,672 

Columbus Traditional 4,729 

Detroit 4,459 

Oklahoma City 6,896 

Portland ^ 4,028 



24.6 


26.3 




-6.7 


25.1 


26.3 


2 *** 


-4.6 


20.1 


23.2 


^2 1 *** 


-13.4 


21.3 


23.2 


9 ♦♦♦ 


-8.3 


19.7 


21.9 


.2 2 *** 


-9.9 


21.0 


23.0 


.2 1 *** 


-8.9 


21.3 


23.0 




-7.5 


18.9 


21.5 


-2 6 *** 


- 12.2 


19.8 


21.5 




- 8.0 


26.2 


27.3 


2 


-3.9 


14.7 


15.6 


_Q 9 *** 


-5.8 


16.3 


20.1 


.3 g 


-18.9 


Average total welfare payments received 






in years 1 to 3 




6,504 


7,058 


-554 *** 


-7.9 


6,578 


7,058 


-480 *** 


- 6.8 


7,853 


9,793 


-1,940 *** 


-19.8 


8,545 


9,793 


-1,247 *** 


-12.7 


11,175 


13,068 


-1,893 *** 


-14.5 


12,122 


14,164 


-2,042 *** 


-14.4 


12,432 


14,164 


-1,732 *** 


- 12.2 


6,072 


7,151 


-1,079 *** 


-15.1 


6,335 


7,151 


-816 *** 


-11.4 


11,538 


11,921 


-383 ** 


-3.2 


4,532 


4,822 


-290 *** 


- 6.0 


7,270 


9,179 


-1,910 *** 


- 20.8 



SOURCE: MDRC calculations from state and county administrative records. 
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Appendix Table D.2 

Impacts on Welfare Receipt in the Last Quarter of Years 1 to 5 



Site and Program 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Atlanta Labor Force Attachment 










Year 1 


78.2 


82.7 


-4 5 *** 


- 5.5 


Year 2 


64.7 


70.9 


-6 2 *** 


- 8.7 


Year 3 


S6.1 


61.2 


2 ♦♦♦ 


- 8.3 


Year 4 


44.S 


50.1 


*7 ♦♦♦ 


-11.3 


Years 


32.6 


3,6.5 


-3 9 ** 


-10.8 


Sample size 


1,441 


1,497 






Atlanta Human Capital Development 










Year 1 


80.8 


82.7 


-2.0 


-2.4 


Year 2 


66.2 


70.9 


-4.8 *** 


-6.7 


Year 3 


S7.4 


61.2 


-3.8 ** 


-6.2 


Year 4 


46.9 


50.1 


-3.2 * 


-6.4 


Years 


34.S 


36.5 


-2.1 


-5.6 


Sanple size 


1,49S 


1,497 






Grand Rapids Labor Force Attachment 










Year 1 


68.8 


77.8 




-11.6 


Year 2 


S3.S 


60.8 


_y 2 *** 


-12.1 


Year 3 


41.2 


48.0 


-6.8 *** 


-14.2 


Year 4 


32.S 


38.0 




-14.6 


Years 


24.3 


27.8 


-3.S ** 


-12.5 


Sample size 


1,SS7 


1,455 






Grand Rapids Human Capital Development 










Year 1 


73.7 


77.8 


-4 2 *** 


-5.4 


Year 2 


S4.3 


60.8 


4 ♦♦♦ 


-10.6 


Year 3 


41.7 


48.0 


-6 3 *** 


-13.1 


Year 4 


32.S 


38.0 




-14.6 


Year S 


2S.8 


27.8 


-2.0 


-7.1 


Sample size 


1,S42 


1,455 






Riverside Labor Force Attachment 










Year 1 


62.9 


69.6 




-9.6 


Year 2 


SO.l 


56.4 


-6.4 *** 


-11.3 


Year 3 


42.7 


47.6 


-4.8 *** 


-10.2 


Year 4 


37.0 


41.1 


2 *** 


-10.1 


Years 


30.2 


34.6 


.4 4 ♦♦♦ 


-12.8 


Sample size 


3,384 


3,342 
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Appendix Table D.2 (continued) 



Site and Program 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Riverside Labor Force Attachment 










Lacked high school diploma or basic skills 










Year 1 


66.4 


72.3 


-6.0 *** 


-8.3 


Year 2 


54.2 


60.0 


-5.8 *** 


-9.7 


Year 3 


47.1 


52.1 


-5.0 *** 


-9.6 


Year 4 


41.5 


46.3 


.4 7 ♦♦♦ 


-10.2 


Year 5 


34.5 


39.0 


_4 ^ ««« 


-11.6 


Sample size 


1,586 


1,539 






Riverside Human Capital Development 










Year 1 


68.2 


72.3 


.4 j ♦♦♦ 


-5.7 


Year 2 


55.9 


60.0 


-4.1 ♦♦ 


-6.8 


Year 3 


46.2 


52.1 


9 ♦♦♦ 


-11.4 


Year 4 


38.7 


46.3 




-16.3 


Year 5 


33.3 


39.0 




-14.7 


Sample size 


1,596 


1,539 






Columbus Integrated 










Year 1 


68.0 


72.5 


.4 ^ ♦♦♦ 


-6.2 


Year 2 


47.1 


53.8 


-6.8 ♦♦♦ 


-12.5 


Year 3 


33.2 


40.3 


_7 J 


-17.6 


Year 4 


22.3 


27.6 


2 ♦♦♦ 


-19.3 


Year 5 


12.7 


16.5 


-3 8 ’*‘*’*‘ 


-23.1 


San^le size 


2,513 


2,159 






Columbus Traditional 










Year 1 


68.9 


72.5 


-3 6 *** 


-5.0 


Year 2 


49.3 


53.8 


-4.6 *** 


-8.5 


Year 3 


34.9 


40.3 


^ 4(4(4( 


-13.5 


Year 4 


24.7 


27.6 


-3.0 ** 


-10.7 


Year 5 


14.1 


16.5 


-2.4 ** 


-14.3 


Sample size 


2,570 


2,159 






Detroit 










Year 1 


85.4 


86.5 


-1.1 


-1.2 


Year 2 


70.1 


73.7 


^2 5 ♦♦♦ 


-4.8 


Year 3 


56.6 


61.4 


-4.8 *** 


-7.9 


Year 4 


45.1 


48.1 


-3.0 ** 


-6.3 


Year 5 


35.0 


35.2 


-0.2 


-0.5 


Sample size 


2,226 


2,233 
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Appendix Table D.2 (continued) 



Site and Program 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Oklahoma City 










Year 1 


50.0 


53.1 


-3.1 *** 


-5.9 


Year 2 


39.1 


41.0 


-1.9 


-4.6 


Years 


30.3 


32.7 


-2.4 ** 


-7.3 


Year 4 


n/a 


n/a 


n/a 


n/a 


Years 


n/a 


n/a 


n/a 


m/a 


San^le size 


3,430 


3,466 






Portland 










Year 1 


59.8 


68.0 


-8.3 *** 


-12.2 


Year 2 


40.6 


53.3 


-12.7 *** 


-23.9 


Years 


25.8 


38.6 


-12.8 *** 


-33.2 


Year 4 


16.5 


22.5 


-6.0 *** 


-26.7 


Year 5 


12.3 


17.1 


-4.8 *** 


-28.2 


Sample size 


3,529 


499 







SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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Appendix Table D.3 



Impacts on Food Stamp Payments and Receipt in Years 1 to 3 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


Percentage 
Change (%) 




Averaee number of months receivinu Food Stamps 
in years 1 to 3 


Atlanta Labor Force Attachment 


2,938 


28.4 


29.0 


- 0.7 * 


- 2.3 


Atlanta Human Capital Development 


2,992 


28.4 


29.0 


- 0.6 * 


- 2.2 


Grand Rapids Labor Force Attachment 


3,012 


22.6 


25.1 


_2 ^ *** 


- 10.0 


Grand Rapids Human Capital Development 


2,997 


23.5 


25.1 


I ^ HtHtUt 


- 6.1 


Riverside Labor Force Attachment 


6,726 


17.6 


20.0 


_2 5 *** 


- 12.3 


Lacked high school diploma or basic skills 


3,125 


18.9 


21.2 


^2 3 *** 


- 10.9 


Riverside Human Capital Development 


3,135 


19.0 


21.2 


_2 2 *** 


- 10.6 


Columbus Integrated 


4,672 


21.7 


23.7 


-2 0 *** 


- 8.5 


Columbus Traditional 


4,729 


22.3 


23.7 




- 5.9 


Detroit 


4,459 


28.3 


29.4 


Q *** 


- 3.5 


Oklahoma City 


6,896 


19.6 


20.0 


- 0.4 


- 1.9 


Portland 


4,028 


21.4 


23.5 


_2 Q 


- 8.7 




Avera&e total Food Stamps received in years 1 to 3 


Atlanta Labor Force Attachment 


2,938 


7,165 


7,371 


-207 ** 


- 2.8 


Atlanta Human Capital Development 


2,992 


7,302 


7,371 


-69 


- 0.9 


Grand Rapids Labor Force Attachment 


3,012 


4,680 


5,092 


_ 4 I 2 *** 


- 8.1 


Grand Rapids Human Capital Development 


2,997 


4,858 


5,092 


-234 ** 


- 4.6 


Riverside Labor Force Attachment 


6,726 


3,322 


3,875 


-553 *** 


- 14.3 


Lacked high school diploma or basic skills 


3,125 


3,652 


4,218 


-566 *** 


- 13.4 


Riverside Human Capital Development 


3,135 


3,683 


4,218 


-536 *** 


- 12.7 


Columbus Integrated 


4,672 


5,618 


6,312 


-694 *** 


- 11.0 


Columbus Traditional 


4,729 


5,830 


6,312 


-482 *** 


- 7.6 


Detroit 


4,459 


6,646 


6,888 


-241 *** 


- 3.5 


Oklahoma City 


6,896 


4,887 


4,988 


-100 


- 2.0 


Portland 


4,028 


5,182 


5,822 


-640 *** 


- 11.0 



SOURCE: MDRC calculations from state and county administrative records. 
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Appendix Table D.4 



Impacts on Food Stamp Receipt and Payments 
in the Last Quarter of Year 5 



Site and Program 


Sample 

Size 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 




Ever received Food Stamos in last Quarter of vear 5 


Atlanta Labor Force Attachment 


2,938 


51.6 


54.7 


-3.1 * ' 


-5.7 


Atlanta Human Capital Development 


2,992 


52.6 


54.7 


-2.2 


-3.9 


Grand Rapids Labor Force Attachment 


3,012 


27.4 


29.8 


-2.4 


-8.0 


Grand Rapids Human Capital Development 


2,997 


29.1 


29.8 


-0.6 


-2.1 


Riverside Labor Force Attachment 


6,726 


30.4 


34.4 


-4.0 


-11.6 


Lacked high school diploma or basic skills 


3,125 


34.8 


38.6 


-3.8 


-9.8 


Riverside Human Capital Development 


3,135 


33.3 


38.6 


-5 3 


-13.8 


Columbus Integrated 


4,672 


21.5 


26.1 




-17.4 


Columbus Traditional 


4,729 


24.7 


26.1 


-1.4 


-5.2 


Detroit 


4,459 


35.6 


36.1 


-0.5 


-1.4 


Oklahoma City 


n/a 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


30.9 


34.0 


-3.1 


-9.1 






Received Food Stamps but not welfare 








in last Quarter of vear 5 




Atlanta Labor Force Attachment 


1,929 


22.3 


22.7 


-0.4 


-1.8 


Atlanta Human Capital Development 


1,953 


23.6 


22.7 


0.9 


4.1 


Grand Rapids Labor Force Attachment 


1,087 


8.8 


7.3 


1.5 


20.7 


Grand Rapids Human Capital Development 


1,084 


7.8 


7.3 


0.5 


6.4 


Riverside Labor Force Attachment 


6,726 


3.4 


3.2 


0.2 


5.2 


Lacked high school diploma or basic skills 


3,125 


3.8 


3.1 


0.8 


25.0 


Riverside Human Capital Development 


3,135 


3.8 


3.1 


0.8 


24.7 


Columbus Integrated 


3,095 


9.5 


10.3 


-0.7 


-7.1 


Columbus Traditional 


3,100 


11.1 


10.3 


0.8 


8.0 


Detroit 


4,459 


7.9 


8.2 


-0.3 


-4.2 


Oklahoma City 


n/a 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


20.4 


19.0 


1.4 


7.3 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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National Evaluation of Welfare-to-Work Strategies 
Appendix Table E.l 

Impacts on Combined Income in Years 1 to 5 



Site and Program 


Program Control Difference 

Group ($) Group ($) (Impact) 


Percentage 
Change (%) 


Atlanta Labor Force Attachment 










Year 1 


7,409 


7,188 


221 ♦♦ 


3.1 


Year 2 


7,888 


7,531 


357 ♦♦ 


4.7 


Year 3 


8,271 


7,928 


343 ♦ 


4.3 


Year 4 


8,587 


8,364 


223 


2.7 


Year 5 


8,983 


8,976 


7 


0.1 


Sample size 


1,441 


1,497 






Atlanta Human Capital Development 










Year 1 


7,264 


7,188 


76 


1.1 


Year 2 


7,834 


7,531 


303 ♦♦ 


4.0 


Year 3 


8,283 


7,928 


355 ♦♦ 


4.5 


Year 4 


8,805 


8,364 


441 ♦♦ 


5.3 


Year 5 


8,933 


8,976 


-42 


-0.5 


Sample size 


1,495 


1,497 






Grand Rapids Labor Force Attachment 










Year 1 


7,743 


8,037 


-294 


-3.7 


Year 2 


7,723 


7,987 


-264 ♦ 


-3.3 


Year 3 


8,220 


8,391 


-171 


-2.0 


Year 4 


8,485 


8,807 


-323 


-3.7 


Year 5 


8,569 


8,950 


-381 


-4.3 


Sample size 


1,557 


1,455 






Grand Rapids Human Capital Development 










Year 1 


7,777 


8,037 


-260 ** 


-3.2 


Year 2 


7,905 


7,987 


-82 


-1.0 


Year 3 


8,165 


8,391 


-111 


-2.7 


Year 4 


8,413 


8,807 


-395 ♦ 


-4.5 


Year 5 


8,666 


8,950 


-284 


-3.2 


Sample size 


1,542 


1,455 






Riverside Labor Force Attachment 










Year 1 


8,722 


8,706 


16 


0.2 


Year 2 


7,663 


7,983 


-321 ♦♦ 


-4.0 


Year 3 


7,420 


7,735 


-315 ♦♦ 


-4.1 


Year 4 


7,469 


7,730 


-261 


-3.4 


Year 5 


7,655 


7,650 


6 


0.1 


Sample size 


3,384 


3,342 
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Appendix Table E.l (continued) 



Site and Program 


Program Control Difference 

Group ($) Group ($) (Impact) 


Percentage 
Change (%) 


Riverside Labor Force Attachment 










Lacked high school diploma or basic skills 










Year 1 


8,510 


8,635 


. -125 


-1.4 


Year 2 


7,342 


7,865 


-523 *** 


-6.6 


Year 3 


7,064 


7,454 


-390 * 


-5.2 


Year 4 


7,017 


7,332 


-314 


-4.3 


Year 5 


7,096 


7,025 


71 


1.0 


Sample size 


1,586 


1,539 






Riverside Human Capital Development 










Year 1 


8,249 


8,635 


-386 *** 


-4.5 


Year 2 


7,303 


7,865 


-562 *** 


-7.2 


Year 3 


6,997 


7,454 


-457 ** 


-6.1 


Year 4 


6,768 


7,332 


-564 ** 


-7.7 


Year 5 


6,607 


7,025 


-418 * 


-6.0 


Sample size 


1,596 


1,539 






Columbus Integrated 










Year 1 


8,543 


8,932 


-389 *** 


-4.4 


Year 2 


8,663 


8,654 


8 


0.1 


Year 3 


8,686 


8,815 


-129 


-1.5 


Year 4 


8,934 


8,964 


-29 


-0.3 


Year 5 


9,211 


9,112 


98 


1.1 


Sample size 


2,513 


2,159 






Columbus Traditional 










Year 1 


8,766 


8,932 


-166 


-1.9 


Year 2 


8,709 


8,654 


55 


0.6 


Year 3 


8,695 


8,815 


-120 


-1.4 


Year 4 


8,966 


8,964 


3 


0.0 


Year 5 


9,096 


9,112 


-17 


-0.2 


Sample size 


2,570 


2,159 






Detroit 










Year 1 


8,694 


8,673 


21 


0.2 


Year 2 


9,266 


9,121 


145 


1.6 


Year 3 


9,907 


9,779 


128 


1.3 


Year 4 


10,146 


10,056 


90 


0.9 


Year 5 


10,243 


10,056 


187 


1.9 


Sample size 


2,226 


2,233 
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Appendix Table E.l (continued) 



Site and Program 


Program Control 
Group ($) Group ($) 


Difference 

(Impact) 


Percentage 
Change (%) 


Oklahoma City 










Year 1 


5,363 


5,478 


-115 


-2.1 


Year 2 


5,269 


5,349 


-80 


-1.5 


Year 3 ' 


5,368 


5,541 


-173 


-3.1 


Year 4 


n/a 


n/a 


n/a 


n/a 


Years 


n/a 


n/a 


n/a 


n/a 


Sample size 


3,430 


3,466 






Portland 










Year 1 


8,836 


8,964 


-128 


-1.4 


Year 2 


8,700 


8,306 


394 


4.7 


Year 3 


8,723 


8,200 


522 


6.4 


Year 4 


8,666 


8,054 


612 * 


7.6 


Years 


8,752 


8,282 


470 


5.7 


Sample size 


3,529 


499 







SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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Appendix Table E.2 

Impacts on Combined Income in Years 1 to 3 and in the Last Quarter of Year 5 



Site and Program 


Sample 

Size 


Program 
Group ($) 


Control 
Group ($) 


Difference 

(Impact) 


Percentage 
Change (%) 






Average combined income in years 1 to 3 


Atlanta Labor Force Attachment 


2,938 


22,870 


21,967 


903 ** 


4.1 


Atlanta Human Capital Development 


2,992 


22,696 


21,967 


729 ** 


3.3 


Grand Rapids Labor Force Attachment 


3,012 


22,865 


23,729 


-865 ** 


-3.6 


Grand Rapids Human Capital Development 


2,997 


23,140 


23,729 


-589 


-2.5 


Riverside Labor Force Attachment 


6,726 


23,385 


24,123 


-738 ** 


-3.1 


Lacked high school diploma or basic skills 


3,125 


22,451 


23,668 


-1,217 *** 


-5.1 


Riverside Human Capital Development 


3,135 


22,147 


23,668 


-1,521 *** 


-6.4 


Columbus Integrated 


4,672 


24,906 


25,493 


-588 


-2.3 


Columbus Traditional 


4,729 


25,195 


25,493 


-298 


-1.2 


Detroit 


4,459 


27,150 


26,924 


227 


0.8 


Oklahoma City 


8,677 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


25,191 


24,686 


506 


2.0 




Average combined income in last quarter of year 5 


Atlanta Labor Force Attachment 


2,938 


2,155 


2,156 


-1 


0.0 


Atlanta Human Capital Development 


2,992 


2,117 


2,156 


-39 


-1.8 


Grand Rapids Labor Force Attachment 


3,012 


2,041 


2,206 


-165 ** 


-7.5 


Grand Rapids Human Capital Development 


2,997 


2,092 


2,206 


-113 


-5.1 


Riverside Labor Force Attachment 


6,726 


1,857 


1,855 


2 


0.1 


Lacked high school diploma or basic skills 


3,125 


1,703 


1,687 


16 


0.9 


Riverside Human Capital Development 


3,135 


1,556 


1,687 


-131 ** 


-7.8 


Columbus Integrated 


4,672 


2,255 


2,217 


38 


1.7 


Columbus Traditional 


4,729 


2,182 


2,217 


-35 


-1.6 


Detroit 


4,459 


2,456 


2,453 


3 


0.1 


Oklahoma City 


8,677 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


2,125 


1,977 


148 


7.5 



SOURCE: MDRC calculations from state and county administrative records. 
NOTES: See Appendix A. 1 . 
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National Evaluation of Welfare-to-Work Strategies 
Appendix Table E.3 



Impacts on Employment and Welfare Status in the Last Quarter of Year 5 



Site and Program 


Sample 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 






Emoloved and not on welfare 




Atlanta Labor Force Attachment 


2,938 


40.5 


39.9 


0.6 


1.4 


Atlanta Human Capital Development 


2,992 


40.0 


39.9 


0.1 


0.3 


Grand Rapids Labor Force Attachment 


3,012 


43.5 


46.5 


-3.0 * 


-6.4 


Grand Rapids Human Capital Development 


2,997 


45.3 


46.5 


-1.2 


-2.5 


Riverside Labor Force Attachment 


6,726 


28.3 


26.0 


2.3 ** 


8.8 


Lacked high school diploma or basic skills 


3,125 


24.5 


21.3 


3 3** 


15.4 


Riverside Human Capital Development 


3,135 


22.7 


21.3 


1.4 


6.6 


Columbus Integrated 


4,672 


50.0 


48.0 


1.9 


4.0 


Columbus Traditional 


4,729 


48.5 


48.0 


0.5 


1.0 


Detroit 


4,459 


39.4 


40.5 


-1.1 


-2.7 


Oklahoma City 


8,677 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


46.2 


40.4 


5.8 ** 


14.3 






Employed and on welfare 




Atlanta Labor Force Attachment 


2,938 


10.4 


11.6 


-1.1 


-9.7 


Atlanta Human Capital Development 


2,992 


11.3 


11.6 


-0.3 


-2.6 


Grand Rapids Labor Force Attachment 


3,012 


11.5 


12.6 


-1.1 


-8.5 


Grand Rapids Human Capital Development 


2,997 


11.4 


12.6 


-1.2 


-9.3 


Riverside Labor Force Attachment 


6,726 


10.1 


9.5 


0.6 


6.3 


Lacked high school diploma or basic skills 


3,125 


10.8 


9.8 


0.9 


9.3 


Riverside Human Capital Development 


3,135 


11.2 


9.8 


1.3 


13.5 


Columbus Integrated 


4,672 


4.4 


7.0 


-2 6 *** 


-36.9 


Columbus Traditional 


4,729 


5.6 


7.0 


-1 3 ** 


-18.9 


Detroit 


4,459 


15.7 


14.8 


0.9 


6.0 


Oklahoma City 


8,677 


n/a 


n/a 


n/a 


n/a 


Portland 


4,028 


3.9 


4.3 


-0.4 


-8.2 



(continued) 
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Appendix Table E.3 (continued) 

Sample Program Control Difference Percentage 

Site and Program Size Group (%) Group (%) (Impact) Change (%) 

Not employed and on welfare 

22.1 24.9 -2.8 * -11.3 



Atlanta Labor Force Attachment 2,938 

Atlanta Human Capital Development 2,992 

Grand Rapids Labor Force Attachment 3,012 

Grand Rapids Human Capital Development 2,997 

Riverside Labor Force Attachment 6,726 

Lacked high school diploma or basic skills 3,125 

Riverside Human Capital Development 3,135 

Columbus Integrated 4,672 

Columbus Traditional 4,729 

Detroit 4,459 

Oklahoma City 8,677 

Portland 4,028 

Atlanta Labor Force Attachment 2,938 

Atlanta Human Capital Development 2,992 

Grand Rapids Labor Force Attachment 3,012 

Grand Rapids Human Capital Development 2,997 

Riverside Labor Force Attachment 6,726 

Lacked high school diploma or basic skills 3, 125 

Riverside Human Capital Development 3,135 

Columbus Integrated 4,672 

Columbus Traditional 4,729 

Detroit 4,459 

Oklahoma City 8,677 

Portland 4,028 



23.2 


24.9 


-1.8 


-7.0 


12.8 


15.2 


-2.4 ♦ 


-15.8 


14.4 


15.2 


-0.8 


-5.3 


20.1 


25.1 


-5.0 *** 


-20.0 


23.8 


29.2 


-5.4 *** 


-18.6 


22.1 


29.2 


_*7 j ♦♦♦ 


-24.2 


8.3 


9.5 


-1.2 


-13.1 


8.5 


9.5 


-1.0 


-11.0 


19.3 


20.4 


-1.1 


-5.3 


n/a 


n/a 


n/a 


n/a 


8.4 


12.9 


-4.5 *** 


-34.8 


Not employed and not 


on welfare 




27.0 


23.6 


3.4 ** 


14.3 


25.6 


23.6 


2.0 


8.3 


32.2 


25.8 


6.5 *** 


25.1 


28.9 


25.8 


3.1 * 


12.2 


41.5 


39.4 


2.1 * 


5.4 


40.9 


39.7 


1.2 


3.1 


44.0 


39.7 


4.3 ** 


10.9 


37.3 


35.4 


1.9 


5.3 


37.3 


35.4 


1.9 


5.3 


25.6 


24.3 


1.3 


5.2 


n/a 


n/a 


n/a 


n/a 


41.5 


42.5 


-1.0 


-2.3 



SOURCE: MDRC calculations from state and county administrative records. 



NOTES: See Appendix A. 1 . 



National Evaluation of Welfare-to-Work Strategies 
Appendix Table E.4 



Impacts on Receipt of Income for Respondents 
and Other Household Members in the Last Month of Year 5 



Site, Program, and 
Income Received 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Atlanta LFA 










Any household member 


98.1 


98.2 


-0.1 


-0.1 


Earnings 


68.7 


65.5 


3.3 


5.0 


AFDC/TANF 


31.4 


38.6 


-7.2 ♦♦♦ 


-18.7 


Food Stamps 


54.0 


62.4 


-8.4 *** 


-13.4 


Child support 


15.3 


14.5 


0.8 


5.5 


SSI/disability 


14.8 


16.6 


-1.8 


-10.6 


Social Security/pension 


6.8 


7.6 


-0.8 


-10.5 


Other income 


6.4 


4.6 


1.8 


38.1 


Respondent 


93.7 


95.2 


-1.5 


-1.5 


Earnings 


59.7 


55.0 


4.7 


8.6 


AFDC/TANF 


29.2 


36.3 


j 


-19.5 


Food Stamps 


51.5 


59.4 


-7.8 *** 


-13.2 


Child support 


14.2 


13.7 


0.5 


3.7 


SSI/disability 


7.8 


8.1 


-0.3 


-3.8 


Social Security/pension 


2.5 


2.2 


0.3 


13.1 


Other income 


5.2 


4.2 


1.0 


24.2 


Other household member 


42.0 


40.7 


1.3 


3.2 


Earnings 


29.1 


27.6 


1.5 


5.4 


AFDC/TANF 


3.3 


4.0 


-0.7 


-17.4 


Food Stamps 


4.2 


5.2 


-1.0 


-18.9 


Child support 


1.1 


0.8 


0.3 


34.5 


SSI/disability 


7.9 


9.6 


-1.7 


-17.7 


Social Security/pension 


4.9 


5.8 


-0.9 


-14.9 


Other income 


1.1 


0.4 


0.7 


191.0 


Sample size 


519 


552 
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Appendix Table £.4 (continued) 



Site, Program, and 
Income Received 


Program Control Difference 

Group (%) Group (%) (Impact) 


Percentage 
Change (%) 


Atlanta HCD 










Any household member 


97.9 


98.2 


-0.3 


-0.3 


Earnings 


67.2 


65.5 


1.7 


2.6 


AFDC/TANF 


36.7 


38.6 


-1.9 


-4.9 


Food Stamps 


55.2 


62.4 


-7.2 ♦♦ 


-11.6 


Child support 


18.2 


14.5 


3.6 ♦ 


. 24.9 


SSI/disability 


16.0 


16.6 


-0.6 


-3.5 


Social Security/pension 


9.1 


7.6 


1.4 


18.9 


Other income 


4.6 


4.6 


0.0 


-0.8 


Respondent 


93.2 


95.2 


-2.1 


-2.2 


Earnings 


54.5 


55.0 


-0.5 


-0.9 


AFDC/TANF 


34.0 


36.3 


-2.3 


-6.5 


Food Stamps 


52.1 


59.4 


-7.3 ♦♦ 


-12.3 


Child support 


17.9 


13.7 


4.2 * 


30.6 


SSI/disability 


8.1 


8.1 


0.0 


0.4 


Social Security/pension 


2.5 


2.2 


0.3 


14.1 


Other income 


3.7 


4.2 


-0.5 


-12.8 


Other household member 


43.2 


40.7 


2.5 


6.1 


Earnings 


30.2 


27.6 


2.6 


9.4 


AFDC/TANF 


5.4 


4.0 


1.4 


35.1 


Food Stamps 


4.9 


5.2 


-0.3 


-5.8 


Child support 


0.3 


0.8 


-0.6 


-69.5 


SSI/disability 


9.3 


9.6 


-0.3 


-2.6 


Social Security/pension 


6.6 


5.8 


0.9 


15.4 


Other income 


0.9 


0.4 


0.5 


131.3 


Sample size 


594 


552 






(continued) 
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Appendix Table E.4 (continued) 



Site, Program, and 
Income Received 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Grand Rapids LFA 










Any household member 


98.2 


99.2 


-1.0 . 


-1.1 


Earnings 


82.1 


80.3 


1.8 


2.2 


AFDC/TANF 


20.4 


22.2 


-1.8 


-7.9 


Food Stamps 


33.6 


36.4 


-2.8 


-7.8 


Child support 


25.7 


29.0 


-3.4 


-11.6 


SSI/disability 


25.3 


21.7 


3.6 


16.6 


Social Security/pension 


5.1 


7.2 


-2.1 


-28.9 


Other income 


8.2 


7.6 


0.6 


. 7.5 


Respondent 


92.8 


93.5 


-0.8 


-0.8 


Earnings 


68.4 


68.7 


-0.3 


-0.4 


AFDC/TANF 


19.7 


21.2 


-1.5 


-7.0 


Food Stamps 


32.2 


35.0 


-2.8 


-8.0 


Child support 


25.7 


27.5 


-1.8 


-6.6 


SSl/disability 


12.2 


10.4 


1.8 


17.7 


Social Security/pension 


1.2 


1.6 


-0.4 


-22.9 


Other income 


5.9 


6.0 


-0.2 


-2.8 


Other household member 


59.3 


55.8 


3.5 


6.3 


Earnings 


46.8 


43.0 


3.8 


8.9 


AFDC/TANF 


1.2 


1.9 


-0.7 


-36.8 


Food Stamps 


2.4 


2.4 


0.0 


1.0 


Child support 


0.6 


1.6 


-0.9 


-60.1 


SSI/disability 


16.5 


13.1 


3.4 


25.5 


Social Security/pension 


3.8 


6.0 


-2.2 * 


-36.3 


Other income 


2.8 


2.1 


0.8 


38.7 


Sample size 


535 


562 
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Appendix Table E.4 (continued) 



Site, Program, and 
Income Received 


Program Control Difference 

Group (%) Group (%) (Impact) 


Percentage 
Change (%) 


Grand Rapids HCD 










Any household member 


98.1 


99.2 


-1.1 


-1.1 


Earnings 


78.8 


80.3 


-1.5 


-1.9 


AFDC/TANF 


22.7 


22.2 


0.5 


2.3 


Food Stamps 


35.0 


36.4 


-1.5 


-4.0 


Child support 


30.3 


29.0 


1.2 


4.3 


SSI/disability 


27.7 


21.7 


6.0 ♦♦ 


27.6 


Social Security/pension 


3.6 


7.2 


-3.6 ♦♦♦ 


-49.7 


Other income 


6.7 


7.6 


-0.9 


-12.2 


Respondent 


92.9 


93.5 


-0.6 


-0.7 


Earnings 


67.6 


68.7 


-1.1 


-1.7 


AFDC/TANF 


20.5 


21.2 


-0.8 


-3.6 


Food Stamps 


31.9 


35.0 


-3.0 


-8.7 


Child support 


29.0 


27.5 


1.5 


5.4 


SSl/disability 


14.0 


10.4 


3.6 * 


35.1 


Social Security/pension 


0.7 


1.6 


-0.9 


-54.9 


Other income 


5.5 


6.0 


-0.6 


-9.4 


Other household member 


57.0 


55.8 


1.3 


2.3 


Earnings 


40.1 


43.0 


-2.9 


-6.7 


AFDC/TANF 


2.9 


1.9 


1.0 


53.7 


Food Stamps 


3.9 


2.4 


1.5 


64.3 


Child support 


1.8 


1.6 


0.2 


13.2 


SSI/disability 


16.3 


13.1 


3.2 


24.0 


Social Security/pension 


3.0 


6.0 


-3.0 ♦♦ 


-49.8 


Other income 


1.2 


2.1 


-0.8 


-40.1 


Sample size 


547 


562 
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Appendix Table E.4 (continued) 



Site, Program, and 
Income Received 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Riverside LFA 










Any household member 


99.4 


98.8 


0.6 


0.6 


Earnings 


73.7 


69.8 


3.8 


5.5 


AFDC/TANF 


41.6 


49.3 


-7.8 ♦♦♦ . 


-15.8 


Food Stamps 


43.6 


50.9 


-7.3 ♦♦♦ 


-14.4 


Child support 


11.7 


12.5 


-0.8 


-6.1 


SSI/disability 


15.3 


13.9 


1.4 


10.0 


Social Security/pension 


8.8 


8.2 


0.7 


8.3 


Other income 


10.0 


11.0 


-1.0 


-9.5 


Respondent 


92.0 


90.3 


1.7 


1.9 


Earnings 


57.9 


51.7 


6.2 ♦♦ 


12.0 


AFDC/TANF 


38.3 


46.5 


-8.2 ♦♦♦ 


-17.6 


Food Stamps 


39.7 


48.7 


-8.9 ♦♦♦ 


-18.4 


Child support 


10.9 


11.3 


-0.4 


-3.3 


SSI/disability 


8.6 


4.7 


3.8 ♦♦♦ 


81.2 


Social Security/pension 


3.0 


2.2 


0.8 


34.7 


Other income 


7.0 


8.8 


-1.9 


-21.1 


Other household member 


52.1 


56.3 


-4.2 


-7.4 


Earnings 


37.6 


42.2 


-4.7 ♦ 


-11.1 


AFDC/TANF 


4.9 


5.1 


-0.2 


-4.8 


Food Stamps 


5.2 


3.7 


1.6 


43.4 


Child support 


0.8 


1.2 


-0.4 


-31.9 


SSI/disability 


7.5 


10.0 


-2.5 


-24.6 


Social Security/pension 


6.8 


6.9 


-0.1 


-0.7 


Other income 


3.6 


2.5 


1.1 


44.7 


Sample size 


499 


720 
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Appendix Table E.4 (continued) 



Site, Program, and 


Program 


Control 


Difference 


Percentage 


Income Received 


Group (%) 


Group (%) 


(Impact) 


Change (%) 



Riverside LFA-lacked high school diploma or basic skills 



Any household member 


99.4 


98.8 


0.6 


0.6 


Earnings 


69.6 


64.0 


5.6 


8.8 


AFDC/TANF 


50.6 


59.4 


1 

00 

bo 

♦ 

♦ 


-14.9 


Food Stamps 


49.1 


59.6 


-10.5 ♦♦♦ 


-17.6 


Child support 


7.0 


11.0 


-4.0 ♦ 


-36.4 


SSI/disability 


14.0 


14.6 


-0.6 


-4.2 


Social Security/pension 


8.3 


7.3 


1.0 


13.5 


Other income 


6.9 


11.5 


-4.5 ♦♦ 


-39.7 


Respondent 


91.3 


91.2 


0.2 


0.2 


Earnings 


52.7 


45.1 


7.6 ♦♦ 


16.7 


AFDC/TANF 


47.1 


56.4 


-9.3 ♦♦ 


-16.5 


Food Stamps 


45.1 


56.4 


-11.3 ♦♦♦ 


-20.0 


Child support 


6.5 


9.8 


-3.2 


-33.3 


SSI/disability 


10.0 


4.8 


5.1 ♦♦♦ 


106.5 


Social Security/pension 


3.7 


2.2 


1.6 


72.2 


Other income 


3.8 


9.0 


-5.3 ♦♦♦ 


-58.5 


Other household member 


47.3 


53.7 


-6.4 ♦ 


-11.8 


Earnings 


32.6 


39.7 


-7.1 ♦♦ 


-17.9 


AFDC/TANF 


5.4 


5.6 


-0.2 


-3.1 


Food Stamps 


5.5 


4.6 


0.9 


19.2 


Child support 


0.5 


. 1.2 


-0.8 


-60.8 


SSI/disability 


4.9 


10.4 


-5.5 ♦♦♦ 


-52.7 


Social Security/pension 


6.1 


5.4 


0.6 


11.7 


Other income 


3.5 


2.7 


0.8 


30.6 


Sample size 


255 


402 
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Appendix Table E.4 (continued) 



Site, Program, and 
Income Received 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Riverside HCD 










Any household member 


98.6 


98.8 


-0.3 


-0.3 


Earnings 


71.7 


64.0 


7.7 ♦♦ 


12.0 


AFDC/TANF 


51.2 


59.4 


-8.2 ♦♦ 


-13.8 


Food Stamps 


52.2 


59.6 


-7.4 ♦♦ 


-12.4 


Child support 


13.5 


11.0 


2.5 


22.7 


SSI/disability 


14.8 


14.6 


0.2 


1.2 


Social Security/pension 


5.3 


7.3 


-2.0 


-27.2 


Other income 


7.6 


11.5 


-3.9 ♦ 


-33.7 


Respondent 


88.6 


91.2 


-2.6 


-2.9 


Earnings 


54.3 


45.1 


9.1 ** 


20.2 


AFDC/TANF 


49.5 


56.4 


-6.9 ♦ 


-12.2 


Food Stamps 


50.2 


56.4 


-6.1 


-10.9 


Child support 


12.5 


9.8 


2.7 


27.9 


SSI/disability 


6.5 


4.8 


1.6 


34.0 


Social Security/pension 


1.1 


2.2 


-1.1 


-51.2 


Other income 


4.5 


9.0 


-4.5 ♦♦ 


-50.0 


Other household member 


54.0 


53.7 


0.3 


0.5 


Earnings 


39.6 


39.7 


-0.1 


-0.2 


AFDC/TANF 


4.4 


5.6 


-1.2 


-21.0 


Food Stamps 


3.9 


4.6 


-0.8 


-17.0 


Child support 


1.0 


1.2 


-0.2 


-17.6 


SSI/disability 


9.2 


10.4 


-1.2 


-11.2 


Social Security/pension 


4.7 


5.4 


-0.8 


-14.0 


Other income 


3.4 


2.7 


0.7 


27.6 


Sample size 


376 


402 
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Appendix Table E.4 (continued) 



Site, Program, and 
Income Received 


Program Control Difference 

Group (%) Group (%) (Impact) 


Percentage 
Change (%) 


Portland 










Any household member 


97.7 


100.1 


-2.4 ** 


-2.4 


Earnings 


81.0 


79.4 


1.6 


2.0 


AFDC/TANF 


19.1 


17.6 


1.5 


8.3 


Food Stamps 


33.8 


42.3 


-8.4 ♦ 


-20.0 


Child support 


20.2 


21.7 


-1.5 


-6.9 


SSI/disability 


16.3 


10.2 


6.1 * 


60.0 


Social Security/pension 


12.7 


8.7 


4.0 


45.8 


Other income 


7.3 


9.2 


-1.9 


-20.7 


Respondent 


90.1 


89.7 


0.4 


0.4 


Earnings 


62.5 


60.6 


1.9 


3.1 


AFDC/TANF 


17.1 


15.3 


1.8 


11.6 


Food Stamps 


32.0 


37.6 


-5.7 


-15.1 


Child support 


17.7 


21.8 


-4.1 


-19.0 


SSI/disability 


8.2 


6.2 


2.0 


32.0 


Social Security/pension 


0.8 


2.9 


-2.0 * 


-71.1 


Other income 


6.5 


6.8 


-0.3 


-4.3 


Other household member 


60.1 


54.2 


6.0 


11.0 


Earnings 


45.0 


42.5 


2.5 


5.9 


AFDC/TANF 


1.9 


2.6 


-0.7 


-25.5 


Food Stamps 


2.8 


6.8 


-4.1 ** 


-59.6 


Child support 


2.5 


-0.1 


2.6 ** 


-3487.8 


SSl/disability 


9.9 


4.6 


5.3 ** 


116.0 


Social Security/pension 


11.9 


5.8 


6.0 ** 


103.1 


Other income 


1.5 


2.9 


-1.4 


-48.3 


Sample size 


281 


223 







SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A.2. 





National Evaluation of Welfare-to-Work Strategies 



Appendix Table E.5 

Impacts on Employment Status of Respondent and 
Other Household Members in the Last Month of Year 5 



Site and Program 


Sample 

Size 


Program Control 

Group (%) Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 






Only respondent employed 




Atlanta Labor Force Attachment 


1,071 


39.6 


37.9 


1.8 


4.7 


Atlanta Human Capital Development 


1,146 


37.0 


37.9 


-0.9 


-2.4 


Grand Rapids Labor Force Attachment 


1,097 


35.3 


37.3 


-2.0 


-5.4 


Grand Rapids Human Capital Development 


1,109 


38.7 


37.3 


1.4 


3.7 


Riverside Labor Force Attachment 


1,219 


36.1 


27.6 


8.5 *** 


30.9 


Lacked high school diploma or basic skills 


657 


37.1 


24.3 


12.7 *** 


52.4 


Riverside Human Capital Development 


778 


32.1 


24.3 


7.8 ** 


31.9 


Portland 


504 


36.0 


37.0 


-0.9 


-2.6 




Respondent and other household member employed 


Atlanta Labor Force Attachment 


1,071 


20.1 


17.2 


2.9 


17.1 


Atlanta Human Capital Development 


1,146 


17.6 


17.2 


0.4 


2.3 


Grand Rapids Labor Force Attachment 


1,097 


33.1 


31.4 


1.7 


5.5 


Grand Rapids Human Capital Development 


1,109 


28.9 


31.4 


-2.5 


-8.0 


Riverside Labor Force Attachment 


1,219 


21.8 


24.1 


-2.3 


-9.6 


Lacked high school diploma or basic skills 


657 


15.7 


20.8 


-5.2 * 


-24.9 


Riverside Human Capital Development 


778 


22.2 


20.8 


1.3 


6.5 


Portland 


504 


26.5 


23.7 


2.8 


11.8 






Only other household member employed 


Atlanta Labor Force Attachment 


1,071 


9.0 


10.4 


-1.5 


-14.0 


Atlanta Human Capital Development 


1,146 


12.6 


10.4 


2.2 


21.2 


Grand Rapids Labor Force Attachment 


1,097 


13.7 


11.6 


2.1 


18.0 


Grand Rapids Human Capital Development 


1,109 


11.3 


11.6 


-0.4 


-3.2 


Riverside Labor Force Attachment 


1,219 


15.8 


18.1 


-2.4 


-13.2 


Lacked high school diploma or basic skills 


657 


16.9 


18.8 


-1.9 


-10.2 


Riverside Human Capital Development 


778 


17.4 


18.8 


-1.4 


-7.5 


Portland 


504 


18.5 


18.8 


-0.3 


-1.5 



(continued) 





Appendix Table E.5 (continued) 



Sample Program Control Difference Percentage 
Site and Program Size Group (%) Group (%) (Inpact) Change (%) 

Neither respondent nor other household 
member employed 



Atlanta Labor Force Attachment 


1,071 


31.3 


34.5 


-3.3 


-9.4 


Atlanta Human Capital Development 


1,146 


32.8 


34.5 


-1.7 


-4.9 


Grand Rapids Labor Force Attachment 


1,097 


17.9 


19.7 


-1.8 


-9.2 


Grand Rapids Human Capital Development 


1,109 


21.2 


19.7 


1.5 


7.7 


Riverside Labor Force Attachment 


1,219 


26.3 


30.2 


-3.8 


-12.7 


Lacked high school diploma or basic skills 


657 


30.4 


36.0 


-5.6 


-15.7 


Riverside Human Capital Development 


778 


28.3 


36.0 


-7.7 ** 


-21.4 


Portland 


504 


19.0 


20.6 


-1.6 


-7.7 



SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A.2. 
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Supplementary Tables to Chapter 7 



National Evaluation of Welfare-to-Work Strategies 
Appendix Table F.l 



Control Group Outcomes for Selected Measures, by Welfare History 



Site and Program 


Average Total 
Earnings 
Sample in Years 

Size 1 to 5 ($) 


Average Welfare 
Payments 
in Years 
1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 






On welfare for two vears or 


more 


Atlanta Labor Force Attachment 


2,063 


14,067 


11,202 


39,133 


Atlanta Human Capital Development 


2,100 


14,067 


11,202 


39,133 


Grand Rapids Labor Force Attachment 


1,791 


18,507 


14,551 


42,370 


Grand Rapids Human Capital Development 


1,775 


18,507 


14,551 


42,370 


Riverside Labor Force Attachment 


3,510 


10,827 


21,539 


39,971 


Lacked high school diploma or basic skills 


1,831 


7,677 


22,800 


38,504 


Riverside Human Capital Development 


1,841 


7,677 


22,800 


38,504 


Columbus Integrated 


3,392 


22,855 


10,095 


43,925 


Columbus Traditional 


3,415 


22,855 


10,095 


43,925 


Detroit 


3,313 


19,404 


17,040 


48,018 


Oklahoma City 


2,076 


11,771 


n/a 


n/a 


Portland 


2,443 


16,885 


13,370 


40,755 






On welfare 


: for less than two 


vears 


Atlanta Labor Force Attachment 


840 


25,528 


6,969 


42,277 


Atlanta Human Capital Development 


847 


25,528 


6,969 


42,277 


Grand Rapids Labor Force Attachment 


1,219 


23,985 


10,670 


41,847 


Grand Rapids Human Capital Development 


1,215 


23,985 


10,670 


41,847 


Riverside Labor Force Attachment 


3,101 


19,393 


14,624 


39,513 


Lacked high school diploma or basic skills 


1,248 


15,370 


16,303 


37,829 


Riverside Human Capital Development 


1,238 


15,370 


16,303 


37,829 


Columbus Integrated 


806 


33,403 


6,508 


47,620 


Columbus Traditional 


793 


33,403 


6,508 


47,620 


Detroit 


1,015 


24,142 


13,688 


46,658 


Oklahoma City 


2,683 


13,787 


n/a 


n/a 


Portland 


1,494 


26,955 


9,104 


43,050 



SOURCES: MDRC calculations from unemployment insurance (UI) earnings records and AFDC records. 



NOTE: N/a = not applicable. 



National Evaluation of Weifare-to-Work Strategies 
Appendix Table F.2 

Control Group Outcomes for Selected Measures, by Recent Work Experience 



Site and Program 


Average Total 
Earnings 
San^le in Years 

Size 1 to 5 ($) 


Average Welfare 
Payments 
in Years 
1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 




Did not work in year prior to random 


assignment 


Atlanta Labor Force Attachment 


1,869 


11,756 


11,024 


36,191 


Atlanta Human Capital Development 


1,937 


11,756 


11,024 


36,191 


Grand Rapids Labor Force Attachment 


1,527 


15,109 


14,680 


38,649 


Grand Rapids Human Capital Development 


1,489 


15,109 


14,680 


38,649 


Riverside Labor Force Attachment 


4,010 


9,143 


20,198 


36,357 


Lacked high school diploma or basic skills 


2,074 


6,649 


21,772 


35,917 


Riverside Human Capital Development 


2,065 


6,649 


21,772 


35,917 


Columbus Integrated 


2,143 


15,916 


9,984 


36,132 


Columbus Traditional 


2,160 


15,916 


9,984 


36,132 


Detroit 


2,978 


16,065 


17,395 


44,839 


Oklahoma City 


3,910 


8,128 


n/a 


n/a 


Portland 


2,317 


15,927 


13,662 


39,323 




Worked in vear prior to random assignment 


Atlanta Labor Force Attachment 


1,069 


27,313 


8,038 


46,689 


Atlanta Human Capital Development 


1,055 


27,313 


8,038 


46,689 


Grand Rapids Labor Force Attachment 


1,485 


26,380 


11,266 


45,667 


Grand Rapids Human Capital Development 


1,508 


26,380 


11,266 


45,667 


Riverside Labor Force Attachment 


2,716 


23,764 


15,333 


45,088 


Lacked high school diploma or basic skills 


1,051 


19,101 


16,888 


42,797 


Riverside Human Capital Development 


1,070 


19,101 


16,888 


42,797 


Columbus Integrated 


2,529 


33,597 


8,176 


51,434 


Columbus Traditional 


2,569 


33,597 


8,176 


51,434 


Detroit 


1,481 


29,425 


13,953 


53,424 


Oklahoma City 


4,767 


16,544 


n/a 


n/a 


Portland 


1,711 


27,710 


9,056 


45,381 



SOURCES: MDRC calculations from unemployment insurance (UI) earnings records and AFDC records. 
NOTE: N/a = not applicable. 




-393- 




National Evaluation of Welfare-to-Work Strategies 
Appendix Table F.3 



Control Group Outcomes for Selected Measures, by Level of Disadvantage 



Site and Program 


Average Total 
Earnings 
Sample in Years 

Size 1 to 5 ($) 


Average Welfare 
Payments 
in Years 
1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 






Most disadvantaged 




Atlanta Labor Force Attachment 


698 


8,486 


12,244 


35,381 


Atlanta Human Capital Development 


734 


8,486 


12,244 


35,381 


Grand Rapids Labor Force Attachment 


458 


8,622 


17,943 


36,697 


Grand Rapids Human Capital Development 


453 


8,622 


17,943 


36,697 


Riverside Labor Force Attachment 


1,362 


5,044 


23,895 


37,108 


Riverside Human Capital Development 


1,362 


5,044 


23,895 


37,108 


Columbus Integrated 


911 


11,383 


11,557 


34,421 


Columbus Traditional 


901 


11,383 


11,557 


34,421 


Detroit 


1,119 


11,532 


20,072 


44,519 


Oklahoma City 


429 


3,888 


n/a 


n/a 


Portland 


617 


10,715 


16,575 


38,513 






Moderately disadvantaged 




Atlanta Labor Force Attachment 


1,887 


17,542 


9,829 


40,004 


Atlanta Human Capital Development 


1,911 


17,542 


9,829 


40,004 


Grand Rapids Labor Force Attachment 


2,123 


20,668 


12,850 


42,108 


Grand Rapids Human Capital Development 


2,078 


20,668 


12,850 


42,108 


Riverside Labor Force Attachment 


4,298 


15,916 


17,185 


39,417 


Riverside Human Capital Development 


3,049 


15,916 


17,185 


39,417 


Columbus Integrated 


3,155 


26,251 


8,916 


45,276 


Columbus Traditional 


3,236 


26,251 


8,916 


45,276 


Detroit 


3,018 


21,404 


15,413 


47,440 


Oklahoma City 


6,170 


10,914 


n/a 


n/a 


Portland 


2,803 


20,143 


11,896 


41,358 



(continued) 
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Appendix Table ¥.3 (continued) 



Site and Program 


Average Total 
Earnings 
Sample in Years 

Size 1 to 5 ($) 


Average Welfare 
Payments 
in Years 
1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 






Least disadvantaeed 




Atlanta Labor Force Attachment 


353 


35,469 


5,762 


49,746 


Atlanta Human Capital Development 


347 


35,469 


5,762 


49,746 


Grand Rapids Labor Force Attachment 


431 


33,891 


8,139 


48,040 


Grand Rapids Human Capital Development 


466 


33,891 


8,139 


48,040 


Riverside Labor Force Attachment 


1,066 


29,903 


12,281 


46,887 


Columbus Integrated 


606 


44,779 


5,585 


56,893 


Columbus Traditional 


592 


44,779 


5,585 


56,893 


Detroit 


322 


41,675 


10,898 


59,547 


Oklahoma City 


2,078 


20,057 


n/a 


n/a 


Portland 


608 


35,591 


5,875 


48,183 



SOURCES: MDRC calculations from unenqjloyment insurance (UI) earnings records and AFDC records. 
NOTE: N/a = not applicable. 





National Evaluation of Welfare-to-Work Strategies 
Appendix Table F.4 

Control Group Outcomes for Selected Measures, by Ethnicity 





Average Total Average Welfare 


Combined 




Earnings Payments 


Income 




Sample in Years in Years 


in Years 


Site and Program 


Size 1 to 5 ($) 1 to 5 ($) 


1 to 5 ($) 



White. non-Hispanic 



Atlanta Labor Force Attachment 


n/a 


n/a 


n/a 


n/a 


Atlanta Human Capital Development 


n/a 


n/a 


n/a 


n/a 


Grand Rapids Labor Force Attachment 


1,470 


24,279 


11,278 


43,288 


Grand Rapids Human Capital Development 


1,515 


24,279 


11,278 


43,288 


Riverside Labor Force Attachment 


3,464 


15,056 


16,394 


37,123 


Lacked high school diploma or basic skills 


1,245 


10,232 


18,157 


34,423 


Riverside Human Capital Development 


1,208 


10,232 


18,157 


34,423 


Columbus Integrated 


2,161 


24,551 


7,981 


41,388 


Columbus Traditional 


2,204 


24,551 


7,981 


41,388 


Detroit 


481 


18,179 


14,535 


42,377 


Oklahoma City 


5,109 


11,997 


n/a 


n/a 


Portland 


2,754 


19,783 


11,176 


39,941 






Black, 


non-Hisnanic 




Atlanta Labor Force Attachment 


2,791 


17,584 


9,992 


40,319 


Atlanta Human Capital Development 


2,838 


17,584 


9,992 


40,319 


Grand Rapids Labor Force Attachment 


1,214 


17,841 


14,938 


42,055 


Grand Rapids Human Capital Development 


1,158 


17,841 


14,938 


42,055 


Riverside Labor Force Attachment 


1,121 


14,411 


19,593 


41,157 


Lacked high school diploma or basic skills 


501 


10,551 


21,281 


39,362 


Riverside Human Capital Development 


510 


10,551 


21,281 


39,362 


Columbus Integrated 


2,414 


26,572 


9,868 


47,350 


Columbus Traditional 


2,431 


26,572 


9,868 


47,350 


Detroit 


3,836 


20,842 


16,482 


48,429 


Oklahoma City 


2,484 


14,484 


n/a 


n/a 


Portland 


798 


24,242 


13,200 


47,414 



(continued) 





Appendix Table F.4 (continued) 



Site and Program 


Average Total 
Earnings 
Sample in Years 

Size 1 to 5 ($) 


Average Welfare 
Payments 
in Years 
1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 








Hispanic 




Atlanta Labor Force Attachment 


n/a 


n/a 


n/a 


n/a 


Atlanta Human Capital Development 


n/a 


n/a 


n/a 


n/a 


Grand Rapids Labor Force Attachment 


244 


15,515 


14,019 


38,617 


Grand Rapids Human Capital Development 


249 


15,515 


14,019 


38,617 


Riverside Labor Force Attachment 


1,858 


14,847 


20,855 


43,658 


Lacked high school diploma or basic skills 


1,210 


11,590 


21,919 


41,991 


Riverside Human Capital Development 


1,240 


11,590 


21,919 


41,991 


Columbus Integrated 


n/a 


n/a 


n/a 


n/a 


Columbus Traditional 


n/a 


n/a 


n/a 


n/a 


Detroit 


n/a 


n/a 


n/a 


n/a 


Oklahoma City 


392 


11,549 


n/a 


n/a 


Portland 


n/a 


n/a 


n/a 


n/a 



SOURCES: MDRC calculations from unemployment insurance (UI) earnings records and AFDC records. 
NOTE: N/a = not applicable. 
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Appendix Table F.5 

Standard Errors for Impacts on Selected Measures, by Welfare History 



Average Total Average Welfare Combined 
Earnings Payments Income 

Sample in Years in Years in Years 

Site and Program Size 1 to 5 ($) 1 to 5 ($) 1 to 5 ($) 

On welfare for two years or more 



Atlanta Labor Force Attachment 
Atlanta Human Capital Development 

Grand Rapids Labor Force Attachment 
Grand Rapids Human Capital Development 

Riverside Labor Force Attachment 
Lacked high school diploma or basic skills 
Riverside Human Capital Development 

Columbus Integrated 
Columbus Traditional 

Detroit 

Oklahoma City 
Portland 



Atlanta Labor Force Attachment 
Atlanta Human Capital Development 

Grand Rapids Labor Force Attachment 
Grand Rapids Human Capital Development 

Riverside Labor Force Attachment 
Lacked high school diploma or basic skills 
Riverside Human Capital Development 

Columbus Integrated 
Columbus Traditional 

Detroit 

Oklahoma City 
Portland 



2,063 


832 


223 


746 


2,100 


824 


221 


739 


1,791 


1,012 


360 


948 


1,775 


1,017 


361 


953 


3,510 


714 


443 


804 


1,831 


789 


610 


1,010 


1,841 


788 


610 


1,009 


3,392 


848 


211 


800 


3,415 


847 


211 


799 


3,313 


891 


293 


791 


2,076 


634 


n/a 


n/a 


2,443 


1,566 


535 


1,541 



On welfare for less than two years 



840 


1,911 


344 


1,713 


847 


1,903 


342 


1,707 


1,219 


1,416 


400 


1,303 


1,215 


1,423 


402 


1,310 


3,101 


996 


436 


998 


1,248 


1,271 


692 


1,388 


1,238 


1,274 


694 


1,391 


806 


2,214 


362 


2,070 


793 


2,236 


365 


2,091 


1,015 


1,828 


511 


1,608 


2,683 


645 


n/a 


n/a 


1,494 


2,493 


516 


2,315 



SOURCES: MDRC calculations from unemployment insurance (UI) earnings records and AFDC records. 
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National Evaluation of Welfare-to-Work Strategies 
Appendix Table F.6 



Standard Errors for Impacts on Selected Measures, by Recent Work Experience 



Site and Program 


Average Total 
Earnings 
Sample in Years 

Size 1 to 5 ($) 


Average Welfare 
Payments 
in Years 
1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 




Did not work in year prior to random 


assignment 


Atlanta Labor Force Attachment 


1,869 


950 


236 


857 


Atlanta Human Capital Development 


1,937 


931 


231 . 


840 


Grand Rapids Labor Force Attachment 


1,527 


1,019 


399 


992 


Grand Rapids Human Capital Development 


1,489 


1,033 


404 


1,005 


Riverside Labor Force Attachment 


4,010 


645 


411 


750 


Lacked high school diploma or basic skills 


2,074 


711 


574 


953 


Riverside Human Capital Development 


2,065 


713 


576 


956 


Columbus Integrated 


2,143 


1,003 


268 


1,002 


Columbus Traditional 


2,160 


1,001 


267 


1,000 


Detroit 


2,978 


898 


318 


822 


Oklahoma City 


3,910 


471 


n/a 


n/a 


Portland 


2,317 


1,703 


528 


1,664 




Worked in year prior to random assienment 


Atlanta Labor Force Attachment 


1,069 


1,477 


308 


1,309 


Atlanta Human Capital Development 


1,055 


1,491 


311 


1,321 


Grand Rapids Labor Force Attachment 


1,485 


1,322 


360 


1,193 


Grand Rapids Human Capital Development 


1,508 


1,316 


358 


' 1,187 


Riverside Labor Force Attachment 


2,716 


1,142 


465 


1,099 


Lacked high school diploma or basic skills 


1,051 


1,523 


738 


1,547 


Riverside Human Capital Development 


1,070 


1,507 


730 


1,530 


Columbus Integrated 


• 2,529 


1,184 


222 


1,079 


Columbus Traditional 


2,569 


1,177 


221 


1,072 


Detroit 


1,481 


1,566 


420 


1,335 


Oklahoma City 


4,767 


517 


n/a 


n/a 


Portland 


1,711 


2,159 


529 


2,014 



SOURCES: MDRC calculations from unemployment insurance (UI) earnings records and AFDC records. 
NOTE: N/a = not applicable. 
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Appendix Table F.7 

Standard Errors for Impacts on Selected Measures, by Level of Disadvantage 



Site and Program 


Average Total 
Earnings 
San^le in Years 

Size 1 to 5 ($) 


Average Welfare 
Payments 
in Years 
1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 






Most disadvantaeed 




Atlanta Labor Force Attachment 


698 


1,095 


377 


1,056 


Atlanta Human Capital Development 


734 


1,063 


366 


1,025 


Grand Rapids Labor Force Attachment 


458 


1,402 


728 


1,542 


Grand Rapids Human Capital Development 


453 


1,416 


735 


1,558 


Riverside Labor Force Attachment 


1,362 


800 


715 


1,140 


Riverside Human Capital Development 


1,362 


802 


717 


1,143 


Columbus Integrated 


911 


1,143 


429 


1,267 


Columbus Traditional 


901 


1,150 


432 


1,274 


Detroit 


1,119 


1,186 


522 


1,148 


Oklahoma City 


429 


924 


n/a 


n/a 


Portland 


617 


2,352 


1,232 


2,730 






Moderately disadvantaeed 




Atlanta Labor Force Attachment 


1,887 


1,099 


239 


977 


Atlanta Human Capital Development 


1,911 


1,092 


238 


970 


Grand Rapids Labor Force Attachment 


2,123 


1,069 


324 


984 


Grand Rapids Human Capital Development 


2,078 


1,082 


328 


997 


Riverside Labor Force Attachment 


4,298 


799 


376 


810 


Riverside Human Capital Development 


3,049 


1,052 


495 


1,067 


Columbus Integrated 


3,155 


1,044 


212 


967 


Columbus Traditional 


3,236 


1,034 


210 


958 


Detroit 


3,018 


1,021 


312 


891 


Oklahoma City 


6,170 


395 


n/a 


n/a 


Portland 


2,803 


1,653 


446 


1,564 
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Appendix Table F.7 (continued) 



Site and Program 


Average Total 
Earnings 
Sample in Years 

Size 1 to 5 ($) 


Average Welfare 
Payments 
in Years 
1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 






Least disadvantaeed 




Atlanta Labor Force Attachment 


353 


3,121 


495 


2,755 


Atlanta Human Capital Development 


347 


3,143 


498 


2,774 


Grand Rapids Labor Force Attachment 


431 


2,947 


624, 


2,616 


Grand Rapids Human Capital Development 


466 


2,846 


603 


2,526 


Riverside Labor Force Attachment 


1,066 


288 


688 


1,928 


Columbus Integrated 


606 


2,958 


365 


2,687 


Columbus Traditional 


592 


2,990 


369 


2,715 


Detroit 


322 


4,344 


833 


3,661 


Oklahoma City 


2,078 


936 


n/a 


n/a 


Portland 


608 


4,204 


724 


3,845 



SOURCES: MDRC calculations from unenployment insurance (UI) earnings records and AFDC records. 
NOTE: N/a = not applicable. 
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National Evaluation of Welfare-to-Work Strategies 
Appendix Table F.8 

Standard Errors for Impacts on Selected Measures, by Ethnicity 





Average Total Average Welfare 


Combined 




Earnings Payments 


Income 




Sanqjle in Years in Years 


in Years 


Site and Program 


Size 1 to 5 ($) 1 to 5 ($) 


1 to 5 ($) 







White, 


non-Hisoanic 




Atlanta Labor Force Attachment 


n/a 


n/a 


n/a 


n/a 


Atlanta Human Capital Development 


n/a 


n/a 


n/a 


n/a 


Grand Rapids Labor Force Attachment 


1,470 


1,276 


373 


1,177 


Grand Rapids Human Capital Development 


1,515 


1,263 


370 


1,165 


Riverside Labor Force Attachment 


3,464 


847 


406 


888 


Lacked high school diploma or basic skills 


1,245 


1,139 


675 


1,303 


Riverside Human Capital Development 


1,208 


1,157 


686 


1,323 


Columbus Integrated 


2,161 


1,154 


243 


1,099 


Columbus Traditional 


2,204 


1,142 


240 


1,088 


Detroit 


481 


2,328 


768 


2,181 


Oklahoma City 


5,109 


459 


n/a 


n/a 


Portland 


2,754 


1,627 


434 


1,552 






Black, 


non-Hisoanic 




Atlanta Labor Force Attachment 


2,791 


826 


193 


738 


Atlanta Human Capital Development 


2,838 


819 


191 


732 


Grand Rapids Labor Force Attachment 


1,214 


1,196 


430 


1,114 


Grand Rapids Human Capital Development 


1,158 


1,224 


440 


1,140 


Riverside Labor Force Attachment 


1,121 


1,475 


778 


1,572 


Lacked high school diploma or basic skills 


501 


1,682 


1,163 


2,122 


Riverside Human Capital Development 


510 


1,671 


1,155 


2,108 


Columbus Integrated 


2,414 


1,102 


246 


1,024 


Columbus Traditional 


2,431 


1,099 


245 


1,022 


Detroit 


3,836 


858 


273 


755 


Oklahoma City 


2,484 


682 


n/a 


n/a 


Portland 


798 


2,783 


975 


2,729 



(continued) 
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Appendix Table F.8 (continued) 



Site and Program 


Average Total Average Welfare 
Earnings Payments 

Sample in Years in Years 

Size 1 to 5 ($) 1 to 5 ($) 


Combined 
Income 
in Years 
1 to 5 ($) 






Hispanic 




Atlanta Labor Force Attachment 


n/a 


n/a 


n/a 


n/a 


Atlanta Human Capital Development 


n/a 


n/a 


n/a 


n/a 


Grand Rapids Labor Force Attachment 


244 


2,771 


1,027 


2,636 


Grand Rapids Human Capital Development 


249 


2,787 


1,033 


2,651 


Riverside Labor Force Attachment 


1,858 


1,098 


624 


1,143 


Lacked high school diploma or basic skills 


1,210 


1,099 


761 


1,287 


Riverside Human Capital Development 


1,240 


1,085 


751 


1,271 


Columbus Integrated 


n/a 


n/a 


n/a 


n/a 


Columbus Traditional 


n/a 


n/a 


n/a 


n/a 


Detroit 


n/a 


n/a 


n/a 


n/a 


Oklahoma City 


392 


1,607 


n/a 


n/a 


Portland 


n/a 


n/a 


n/a 


n/a 



SOURCES: MDRC calculations from unemployment insurance (UI) earnings records and AFDC records. 
NOTE: N/a = not applicable. 
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Data on participation, degree receipt, job quality, income, transitional benefits, health care 
coverage, child care, child outcomes, and several other measures used in this report come from 
responses to the Five-Year Client Survey, Child Outcomes Study (COS) survey, and Teacher 
Survey. This appendix assesses the reliability of impact results for the three survey samples. It 
also considers whether impacts estimated for sample members who responded to the survey may 
be generalized to the “eligible” sample — that is, to sample members who were randomly as- 
signed during the months when the sample was chosen and who met the criteria for selection. 

The analysis addresses the following questions for each sample: 

• Are the response rates (sample members interviewed as a percentage of sam- 
ple members chosen to be interviewed) high enough to satisfy the usual stan- 
dards of impact analysis? 

• Are differences in response rates across research groups small enough to indi- 
cate that comparisons between those groups will yield unbiased impact esti- 
mates? 

• Are impact estimates based on unemployment insurance earnings records and 
welfare payment records similar for respondents and the eligible samples? 

For respondents in the Client Survey and COS survey samples, the findings from these 
multiple tests were generally, but not entirely, positive. For the LFA and FICD programs in At- 
lanta and Grand Rapids, impact results for respondents appear reliable and representative of ef- 
fects for members of the eligible sample. Somewhat greater caution is required in generalizing 
about the impacts for Riverside LFA survey respondents, especially for the COS sample because 
(1) COS response rates in Riverside were relatively low and (2) relatively large differences were 
found when comparing five-year impacts on welfare payments and (for the COS) total earnings 
for the respondent and eligible samples. More extreme disparity was found in comparing impacts 
for the eligible and respondent samples for Portland (Client Survey) and for Riverside HCD 
(both surveys). These results suggest that impacts on survey outcomes for Riverside HCD and 
Portland are not representative of effects for the eligible sample. It is also uncertain whether re- 
sults from the Teacher Survey represent program effects for the eligible sample because of low 
response rates in all sites and, for several programs, inconsistency across samples in estimated 
earnings and welfare impacts. 



I. Key Analysis Samples 

The response analysis involves comparing background characteristics and impact results 
for the following samples drawn from the full research sample: 

The survey-eligible sample (“eligibles”). Sample members in the full research 
sample who were randomly assigned during months in which the survey sample 
was selected and who met the criteria for inclusion. 

The fielded sample ("fieldeds ”). Members of the eligible sample who were chosen 
to be interviewed. 
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The respondent sample ("respondents Members of the eligible sample, chosen 
to be interviewed (that is, fieldeds), who were interviewed. 

The nonrespondent sample ("nonrespondents Members of the eligible sample, 
chosen to be interviewed (that is, fieldeds), who were not interviewed. They could 
not be located or declined to be interviewed. 

II. Survey Selection and Sampline Ratios 

As discussed in Chapter 2, in Atlanta, Grand Rapids, and Riverside, the samples for the 
Five-Year Client Survey, COS survey, and Teacher Survey were nested: COS sample members 
make up a part of the larger Client Survey sample, and teacher surveys were collected for a por- 
tion of the COS sample. 

A. The Eli£ible Samples 

With very few exceptions, the eligible samples originally chosen for the Two-Year Client 
Survey and COS survey were the same for the later surveys.' Sample members were randomly 
assigned during some, but not all, months of sample intake. (See Chapter 2, Table 2.2.) Limiting 
the eligible sample in this way can introduce “cohort effects”: impact estimates that are espe- 
cially large or small for sample members randomly assigned during particular months. A cohort 
effect may occur because members of the survey-eligible sample differ in measured or unmeas- 
ured background characteristics from persons randomly assigned in other months. Changes in 
area labor markets or in program implementation that occur at some point after the start-up of 
random assignment may also introduce cohort effects — for example, by increasing or decreasing 
a program’s relative success in moving welfare recipients from welfare to work. In addition, the 
research strategy in Atlanta, Grand Rapids, and Riverside required exclusion of sample members 
with certain background characteristics: teen parents, parents with children under age 3 (in At- 
lanta and Riverside), men with children aged 3 to 5, people who did not speak either English or 
Spanish, and people who did not provide information on their education status and children’s 
ages prior to random assignment. Survey “eligibles” in Atlanta, Grand Rapids, and Riverside 
who were also eligible for selection to the COS sample included women with at least one child 
aged 3 to 5 at the time of random assignment. 

Differences of moderate size — that is, not enough to change the overall findings about a 
program — were found for the Atlanta LFA and HCD, Grand Rapids LFA, and Portland pro- 
grams, when impacts for the survey- eligible and full samples were compared. For instance, five- 
year earnings impacts for both Atlanta programs were about $1,000 larger when calculated for 
the eligible sample rather than the full sample. Earnings gains averaged about $700 more for 
LFA-eligibles in Grand Rapids. In contrast, impacts for program group members in the Portland 



'See Freedman et al., 2000a, Appendix E, for a response analysis of the Two-Year Client Survey. A small num- 
ber of sample members who were chosen to be surveyed were subsequently discovered to have background charac- 
teristics (such as nonproficiency in English or Spanish) that made them ineligible. These sample members were 
dropped from the eligible samples for the survey at five years. In addition, it was decided to drop single fathers from 
the Five-Year Client Survey eligible sample, although some were interviewed at two years. 
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survey-eligible sample fell about $900 below the increase for the full sample. Differences in im- 
pacts were smaller for Riverside LFA and HCD and Grand Rapids HCD. 

B. The Fielded Samples 

The fielded samples for the Five-Year Client Survey and COS survey were selected from 
among the sample members originally fielded for the two-year surveys. Sample selection for 
these earlier surveys occurred in the following way: The eligible samples in Atlanta, Grand Rap- 
ids, Riverside were divided into strata according to sample members’ research group, date of 
random assignment, age of youngest child, and pre-random assignment educational attainment. 
In Portland, the strata were defined by sample members’ research group and date of random as- 
signment. For research purposes, different sampling ratios, ranging from 16 to 100 percent, were 
used in selecting members of the fielded sample from within each stratum. (The sampling ratio is 
the percentage of eligible sample members selected.) Sample members were chosen at random 
within each stratum. Although corrected for, as discussed below, differences in sampling ratios 
may affect survey impact estimates. For instance, unless the total sample size is large, different 
sampling ratios increase the likelihood that persons chosen in one research group differ (perhaps 
in unmeasured characteristics) from persons chosen in another research group. 

In Portland and Grand Rapids all sample members fielded at two years were again 
fielded at five years, whereas in Atlanta and Riverside funding limitations made it necessary to 
select a subsample from the original fielded sample. For research purposes, MDRC gave prior- 
ity to members of the Two-Year COS survey fielded sample in selecting the Five-Year Client 
Survey sample. 

Members of the five-year fielded samples included respondents and nonrespondents to 
the earlier survey interviews. In addition, the boy or girl chosen as the “focal child” for the two- 
year COS survey continued as the focal child for the five-year study. 

By the strictest definition, the fielded sample for the Five-Year COS survey also consti- 
tuted the fielded sample for the Five-Year Teacher Survey, because researchers were prepared to 
interview teachers of every COS focal child. However, surveys were attempted only for COS fo- 
cal children whose mother was interviewed in person at five years and who then gave her written 
permission to contact her child’s school. Thus, an alternative definition of the fielded sample for 
the Teacher Survey would limit the sample to focal children of COS survey respondents, or even 
to children whose mothers signed a permission form during their interview. 

C. Survey Respondents 

Survey respondents are members of the fielded sample who completed an interview (or a 
sufficient portion to be usable for research). The concept is straightforward for the Five-Year 
Client Survey, but not for the COS survey. Sample members fielded only for the Client Survey 
could be interviewed by phone or in person. However, sample members fielded for the COS sur- 
vey were supposed to answer the Client Survey and additional COS survey questions during an 
in-person session that also included observations of interactions between the COS mother and 
focal child, an interview with the focal child, and administration of a standard assessment of the 
focal child’s intellectual development. As noted in Chapter 2, 203 mothers in the COS fielded 
sample had moved too far away to be visited by interviewers or could not participate a COS in- 




ij.. . 
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person interview for other reasons. They did, however, answer the Client Survey by phone, in- 
cluding the questions on child care and child outcomes asked of all respondents to the Client 
Survey. These sample members are counted as respondents to the Client Survey, but not as re- 
spondents to the COS survey. Thus, the COS respondent sample is limited to mothers who par- 
ticipated in the in-person interviews, observations, and assessments. Finally, for this analysis, the 
“respondent” sample for the Teacher Survey includes COS survey respondents with a completed 
Teacher Survey about the focal child. 

III, Weighting 

For this report, weights were applied to the survey respondent sample to correct for dif- 
ferences in sampling ratios between the strata. In the unweighted fielded survey sample in these 
sites, strata (that is, sample members who share background characteristics and have the same 
sampling ratio) with high sampling ratios are overrepresented and strata with low sampling ratios 
are underrepresented. To make the fielded sample more closely replicate the background charac- 
teristics of survey eligibles, weights for each stratum were set to equal the inverse of the sam- 
pling ratio for that stratum. For example, a stratum in which 1 eligible person in 4 was chosen 
would receive a weight of 4 (or 4/1), whereas a stratum in which every eligible person was cho- 
sen would receive a weight of 1 (or 1/1). The same weights are used for the respondent sample. 

It should be noted that under some conditions impacts for a weighted respondent sample 
may still be different from those for the eligible sample. For example, this result could occur if 
very different proportions of program and control group fieldeds answered the survey, or if 
members of a subgroup within one research group were more likely to be interviewed than their 
counterparts in a different research group. These issues are addressed in the next section. 

IV. Response Rates 

Table G.l shows the response rate, the percentage of the fielded sample who responded, 
by survey sample, program, and research group. The goal of each survey effort was to obtain re- 
sponses from at least 70 percent of the fielded sample in every research group. For the Five-Year 
Client Survey, response rates exceed 70 percent for all programs and research groups (and 80 
percent in Atlanta and Grand Rapids) and are high enough to suggest that the survey probably 
represents the eligible sample. These results inspire confidence in the impacts for respondents. 




-409- 






447 



National Evaluation of Welfare-to-Work Strategies 
Appendix Table G.l 

Number of Fielded Survey Sample Members and Client Survey Response Rates 
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Response rates also exceed the 70 percent threshold for the COS sample fielded in At- 
lanta and Grand Rapids. In Riverside, however, COS response rates fell below this standard, 
ranging from about 63 percent (LFA) to 67 percent (HCD).^ Therefore, greater caution is re- 
quired when interpreting results for the COS sample in Riverside. 

Response rates were much lower for the Teacher Survey, using the strictest definition of 
the fielded sample: about 40 percent for Riverside, 50 percent for Atlanta and Grand Rapids 
HCD, and 58 percent for Grand Rapids LFA and control group members. In each site, response 
rates increase about 10 percentage points when COS survey respondents constitute the fielded 
sample. (Results not shown.) These data should be considered the least reliable indicators of pro- 
gram effects on children. 

V. Research Group Differences in Response Rates 

Different response rates among research groups can be a potential source of bias in re- 
search group comparisons. Such differences suggest that research groups may differ by unob- 
servable characteristics that cannot be controlled for and may affect impact estimates. The results 
for the Five-Year Client Survey fielded samples indicate that within each site response rates for 
each research group differ by 5 percentage points or less, which should be interpreted as a good 
result. (See Table G.l.) Variation in response rates was only slighter greater for the smaller COS 
survey and Teacher Survey fielded samples. 

VI. Research Group Differences in Background Characteristics 

An additional concern when estimating impacts from survey responses is that research 
groups may differ in background characteristics that affect future employment, welfare receipt, 
and other outcomes. Differences in these observable characteristics can be corrected for in the 
regression impact model and do not pose a large problem. These differences, however, may indi- 
cate variation in unobservable characteristics that as noted above, carmot be controlled for in the 
impact analysis. The following results show that background characteristics differ by research 
group in three programs. 

To determine whether there are any observable program-control differences within the 
survey respondent sample, the 0/1 dummy variable indicating membership in the program group 
was regressed on pre-random assignment demographic information for the fielded and the re- 
spondent samples. A statistically significant p-value of the R-square of the regression described 
above indicates that research groups have different background characteristics and that greater 
caution is required in interpreting impact results. The results show that differences in demo- 
graphic characteristics are evident in Atlanta HCD, Riverside LFA, and Portland among respon- 
dents to the Client Survey and in Riverside LFA among respondents to the COS survey. How- 
ever, this problem was not severe. Even in programs for which program-control group differ- 



^In Riverside, response rates would exceed 75 percent for all research groups if the COS mothers who answered 
only the Client Survey were counted as respondents. 
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ences were found, these differences rarely exceeded 5 percentage points.^ (Results not shown.) 
No statistically significant differences were found for respondents to the Teacher Survey. (Re- 
sults not shown.) 

VII. A Comparison of Impacts for Survey Respondents 
and the Eligible Samples 

Impacts on five-year earnings and welfare payments based on administrative records were 
estimated for the eligible and respondent samples for the Client Survey, COS survey, and Teach- 
er Survey. The results are summarized in Figures G.1-G.3. In these figures, impacts for the 
eligible sample (weighting not required) are compared with the weighted impacts for the respon- 
dent sample. Programs that fall near the 45-degree line that is drawn on these figures have similar 
impacts for the respondent sample and the eligible sample, whereas programs that fall well above 
or below the 45- degree line show large variation in impacts. Similarity in results suggests that 
impacts estimated with survey data for respondents represent the effects that would be found for 
the eligible sample — especially on measures that are often affected by employment and welfare 
levels, such as child care use, health care coverage, and child outcomes. Differences in impact 
estimates suggest the opposite, though some variation in impacts should be expected because of 
differences in sample sizes. 

For these comparisons a problematic result is considered to have occurred when the im- 
pact for respondents exceeded or fell below the impact for the eligible sample by an amount suf- 
ficient to change the findings. For example, results would be problematic if a program led to an 
unusually large impact on total earnings or welfare payments for the eligible sample, based on 
the range of impacts found in previous evaluations of welfare-to-work programs, but to an un- 
usually small impact when calculated for the respondent sample (or vice versa).'* Similarly, 
within a site, results would be problematic if the relative effects of LFA and HCD differed sub- 
stantially when calculated for the two samples. Under these circumstances, findings on other out- 
comes may not represent the likely impact for the larger sample. 



^An important exception occurred in Portland, where whites made up about 63 percent of program group re- 
spondents but 71 percent of control group respondents. As noted in Chapter 7, impacts on earnings were much larger 
for whites than for African-Americans among members of the full sample. 

^More specifically, programs are considered to have led to a “large” impact on earnings if program group mem- 
bers’ average earnings exceeded the control group average by $900 or more per year. Earnings inq)acts of between 
$300 and $900 per year are considered “moderate”; and impacts of between $100 and $300 are considered “small.” 
Programs that lead to earnings impacts of less than $100 per year are considered to have resulted in no intact. Thus, 
in comparing impacts for the respondent and eligible samples, a difference of impacts in earnings of $600 or more 
per year, or $3,000 or more over five years, could change a finding of “small” impacts to “large” impacts, or vice 
versa. This difference in impacts would be considered problematic. In contrast, a variation in impacts of $1,000 or 
less over five years would be unlikely to change the overall findings about a program’s relative success in increasing 
earnings. Differences in impacts of between $1,000 and $3,000 may also cause concern, particularly if one sample 
shows no effects on earnings and the other sample shows a moderate gain or loss compared with the control group. 

Reductions in welfare expenditures of 10 percent or more below the control group average may be considered 
“large,” whereas reductions of between 2 and 5 percent may be considered “small.” 
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A. Comparing Impacts for the Client Survey Eligible and Respondent Samples 

For the Client Survey eligible and respondent samples, five-year earnings impacts were 
nearly identical for the Riverside LFA program and relatively close (within $450 to $550) for the 
Atlanta and Grand Rapids HCD programs (Figure G.l). Impact estimates for the two samples 
varied by a somewhat larger amount for Atlanta and Grand Rapids LFA ($800 to $900). For both 
programs, impacts were smaller for the respondent sample, but not by enough to change the 
overall conclusions that the two programs led to moderately large impacts on earnings.^ 

Earnings impacts differed dramatically and are problematic for Riverside HCD and for 
Portland. For the eligible sample, Portland led to a substantial impact on cumulative earnings 
(averaging about $850 per year) for the full impact sample, but only a small effect (of about $225 
per year) for the respondent sample.^ The opposite result occurred for Riverside HCD. Five-year 
earnings impacts for the respondent sample (of about $700 per year) greatly exceed impacts es- 
timated for the eligible sample (about $180 per year). Furthermore, impacts for HCD respondents 
in Riverside also exceed effects for LFAs, a very different result from what was found for the full 
sample.’ For both of these programs, these differences in impacts change the conclusion about 
program effects and create considerable doubt as to whether other effects estimated with survey 
data may be generalized to the eligible sample.* 

Comparisons of impacts on total welfare payments show greater consistency between the 
eligible and respondent samples. The Riverside LFA and Portland programs led to noticeably 
larger reductions in average welfare payments over five years when estimated for the respondent 
sample. However, reductions for the eligible samples in the programs averaged more than 15 
percent below control group levels. Therefore, these differences in impacts do not change the 
overall finding (discussed in Chapter 5) that Riverside LFA and Portland programs led to large 
decreases in welfare payments.^ 



^By standards described in footnote 4, earnings impacts would be characterized as moderate for Grand Rapids 
LFAs in the eligible sample, but as small for LFAs in the respondents saitqile. However, the change in impacts is not 
that large: from about $450 per year for the eligible sample and about $290 per year for the respondent saitqile. 

^Earnings impacts were only slightly smaller for the eligible sample in Portland than for the full saitqile. (Results 
not shown.) 

’Earnings impacts for Riverside LFA and HCD eligible sanqiles are much closer to impacts estimated for the full 
samples. (Results not shown.) 

®In Portland, two-year impact estimates were much closer for the eligible and respondent samples. See Freedman 
et al., 2000, Appendix E. 

®For Portland, there is also a large discrepancy in impacts on combined income for the eligible sanqile (+$1,600) 
and respondent sample (-$3,900). This result shows again that findings for respondents on survey outcomes are 
likely not representative of impacts for the eligible (or full) samples. The difference in impacts on combined income 
was also substantial for Riverside HCD (-$3,000 eligibles; -$500 respondents) and Riverside LFA (-$1,300 eligibles; 
-$3,300 respondents). (Results not shown.) 
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National Evaluation of Welfare-to-Work Strategies 
Appendix Figure G.l 

Five-Year Earnings and Welfare Impacts for 
Five-Year Client Survey Respondents and Eligibles 



Earnings 




Welfare 




Survey eligibles 

SOURCE: MDRC calculations from the Five-Year Client Survey. 
NOTES: See Appendix A. 2. 




- 414 - 



452 



B. Comparing Impacts for the COS Survey Eli 2 ible and Respondent Samples 



1. COS Respondents 

Five-year earnings impacts were very consistent for Atlanta and Grand Rapids HCD 
when calculated for the COS eligible and respondent samples. Estimated impacts for Grand Rap- 
ids LFA differed to a greater extent (about $1,500 higher for respondents) but still support a con- 
clusion that these programs led to a moderate impact on earnings. Impacts varied more dramati- 
cally for the two programs in Riverside, especially for Riverside HCD. For COS survey respon- 
dents in Riverside, each program led to unusually large gains in earnings above control group 
levels that averaged more than $1,200 per year. For Riverside LFA, this change in impacts is less 
of a problem, because LFAs in the eligible sample averaged close to $800 per year more than 
control group members, an impact that few programs have achieved for parents of young chil- 
dren. For Riverside HCDs, however, estimated earnings gains more than tripled when calculated 
for COS survey respondents and lead to a conclusion that the program was much more successful 
than suggested by impacts for the eligible sample. 

Less variation was found in comparing five-year impacts on welfare payments for the 
COS survey eligible and respondent samples (Figure G.2, lower panel). Reductions in welfare 
dollars were noticeably different for the respondent sample only for Riverside LFA. However, 
the impact for the eligible sample was already large for Riverside LFA. 

2. Teacher Survey Respondents 

As noted above, the “respondent” sample for the Teacher Survey is made up of the moth- 
ers in the COS survey respondent sample whose focal child was the subject of a completed 
Teacher Survey. For most sites and programs, the comparison of five-year impacts on total earn- 
ings and welfare payments yielded similar results as the comparison for COS eligibles and all 
COS respondents discussed above (Figure G.3). The main exception is the Grand Rapids LFA 
program, which led to earnings impacts that averaged nearly $1,000 per year for Teacher Survey 
respondents, but only $350 per year for COS survey eligibles. In addition, the Grand Rapids 
HCD program resulted in a large decrease in welfare payments for COS survey eligibles, but only 
a small reduction for Teacher Survey respondents." 



‘°The same degree of variation in earnings inpacts was also seen for Riverside LFAs in need of basic education. 
(Results not shown.) 

"The discrepancy in earnings impacts is less extreme for Grand Rapids LFA when COS respondents are consid- 
ered as the eligible sample: $1,000 per year above the control group compared with about $650 per year. For Grand 
Rapids HCD, however, impact estimates were only slightly closer when COS respondents were compared with 
Teacher Survey respondents. 
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National Evaluation of Welfare-to-Work Strategies 
Appendix Figure G.2 

Five-Year Earnings and Welfare Impacts for 
Five-Year Child Outcomes Study Respondents and Eligibles 



Earnings 




Welfare 




Survey eligibles 

SOURCE: MDRC calculations from the Five-Year Child Outcomes Study survey. 



NOTES: See Appendix A.2. 



National Evaluation of Welfare-to-Work Strategies 
Appendix Figure G.3 

Five-Year Earnings and Welfare Impacts for 
Five-Year Teacher Survey Respondents and Eligibles 




Welfare 




SOURCE: MDRC calcularions from the Five-Year Teacher Survey. 
NOTES: See Appendix A.2. 
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A Comparison of Impacts Estimated from Survey 
and UI Earnings Data 
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Employment and earnings impacts in this report are estimated from statewide-automated 
unemployment insurance (UI) earnings records and from responses to the Two-Year Client Sur- 
vey and Five-Year Client Survey. This appendix compares employment impacts from these 
sources over five years and during year 5 and investigates why they differ in some programs. The 
results demonstrate that surveys sometimes recorded jobs that were missed by statewide UI earn- 
ings reporting systems and other times underreported employment. Further, in some sites pro- 
gram and control groups varied in the degree to which employment was underreported on the 
survey or with UI records. 



I. Possible Reasons for Differences Between Survey and UI Earnings Data 

Survey data are self-reported. They include jobs that are not covered or not reported to the 
state UI system, such as self-employment, some domestic work, federal government or military 
jobs, informal employment, or out-of-state jobs. UI earnings data, however, may include jobs that 
respondents fail to recall or are reluctant to report on the survey. Survey respondents may also have 
had problems recalling start and end dates of some jobs, particularly those that started early in the 
follow-up and lasted for a short period of time. On the other hand, some employers may have de- 
layed reporting employment to the UI system until after the files were created for this report.' 

Furthermore, survey and UI earnings data presented in this report cover somewhat differ- 
ent time periods. UI earnings data were available for each quarter of follow-up, whereas the two 
survey interviews recorded employment in a less complete way. The two-year survey collected 
information on up to six jobs during years 1 and 2, but the five-year survey recorded only infor- 
mation about respondents’ current or most recent job.^ In addition, the indicator of “ever em- 
ployed during years 1 to 5” used in the report covers somewhat different follow-up periods, when 
measured with UI earnings or survey data. UI data cover quarters 2-21 after random assignment, 
whereas the follow-up period for survey data extended from the random assignment month to the 
interview date, which for some respondents occurred several months after the end of year 5.^ 
Moreover, year 5 survey impacts cover months 48-60, starting and ending slightly earlier than the 
follow-up for UI earnings for most sample members."* 



'in addition, sample members’ Social Security number is needed to match to UI earnings. Some Social Security 
numbers are reported or recorded incorrectly at random assignment. Sometimes the incorrect number does not match 
to any UI earnings records; other times it matches to the records for another person. 

^In addition, 434 respondents to the Five-Year Client Survey were not interviewed after two years. 

^As noted in the report, UI data are recorded quarterly. Quarter 1 may include earnings from before random as- 
signment and is therefore excluded from the analysis of program impacts. Quarters 2-2 1 correspond to months 2 to 
61, 3 to 62, or 4 to 63 after random assignment, depending on whether sample members were randomly assigned 
during the first, second, or third month of a calendar quarter. 

‘'Year 5 as measured with UI earnings records includes quarters 18-21, which correspond to months 50 to 61, 51 
to 62, or 52 to 63, depending on the respondent’s month of random assigrunent. 
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II. Reporting Discrepancies for Sample Members with Both Survey 
and UI Earnings Data 

One potential source of differences in impact estimates from survey and UI earnings data 
is discrepant reporting. To see if this were a problem, for each sample member in the survey re- 
spondent sample, employment reported any time after random assignment from the survey data 
was directly compared with UI earnings recorded during quarters 2-21 (or years 1 to 5).^ A sec- 
ond comparison was made for year 5. The results are shown in Table H.l . 

For each comparison, a match occurred if both sources recorded earnings during at least 
one quarter of follow-up or if neither source recorded earnings during any quarter. Match rates 
over five years ranged from 82.6 percent in Riverside to 91.7 percent in Grand Rapids when pro- 
gram and control group members were considered together. As shown in Table H.l, patterns of 
discrepancies differed by site. In Atlanta and Grand Rapids, relatively few mismatches occurred. 
In Atlanta, neither source recorded a higher incidence of employment, and in Grand Rapids the 
employment level was only slightly higher when recorded with survey data. Discrepancies in re- 
ported employment occurred most often in Portland and Riverside, and nearly all mismatches 
resulted from employment that was recorded only on the survey. 

Results for year 5 show a similar pattern, except that the overall match rates were lower 
for each site, ranging from 73.1 percent in Portland to 81.5 percent in Atlanta. As with the five- 
year employment measure, in Atlanta and Grand Rapids about the same percentage of respon- 
dents had information on employment recorded only on UI earnings records as had information 
recorded only from their survey responses. However, in Portland and, especially, in Riverside 
survey responses provided considerably higher rates of employment than UI records. 



III. Observed Patterns of Differences Between Survey 
and UI Earnings Impacts 

Table H.2 compares the incidence of employment in years 1 to 5 for program and control 
group survey respondents in each program, as well as program impacts, estimated from UI earn- 
ings (row 1: records impact) and survey responses (row 2: survey impact). A comparison of these 
two rows highlights the difference in estimates from survey and UI earnings data. 

From the standpoint of consistency, the preferred result would be (by definition) for both 
sources to record the same information for each person. This result occurred in Atlanta, where 
survey responses and UI records captured the same employment levels for LFAs, HCDs, and 
control group members, as well as similar impacts. Results for Grand Rapids were nearly as 
good. For all three research groups, survey responses display employment levels that were between 



^Some sample members were interviewed after the follow-up period for UI earnings and were excluded from 
this comparison. 
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National Evaluation of Welfare- to-Work Strategies 
Appendix Table H.l 

Percentage of Survey Sample Members Having Earnings on 
Survey or UI Records, but Not on Both 



Site and Program 


Five-Year 
Survey Only (%) 


UI Earnings 
Data Only (%) 






Years 1 to 5 


Atlanta 


5.9 


6.2 


Grand Rapids 


5.9 


2.4 


Portland 


12.9 


1.5 


Riverside 


15.3 


2.1 






Years 


Atlanta 


8.5 


10.0 


Grand Rapids 


11.1 


8.7 


Portland 


17.4 


9.5 


Riverside 


18.5 


5.7 



SOURCES: MDRC calculations from the Five-Year Client Survey and unemployment insurance (UI) 
records. 



NOTES: See Appendix A. 2. 



National Evaluation of Welfare-to-Work Strategies 
Appendix Table H.2 

Comparison of Impact Estimates from Survey and UI Earnings 
Data for Employment in Years 1 to 5 



Site and Program 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Impact) 


Percentage 
Change (%) 


Atlanta Labor Force Attachment 
Records impact: survey sample 


84.3 


Ever emploved in years 1 to 5 
84.6 -0.3 


-0.4 


Survey impact: survey sample 


83.3 


82.9 


0.4 


0.4 


Atlanta Human Capital Development 


Records impact: survey sample 


83.4 


84.6 


-1.2 


-1.4 


Survey impact: survey sample 


81.2 


82.9 


-1.7 


-2.0 


Grand Rapids Labor Force Attachment 


Records impact: survey sample 


90.9 


88.4 


2.5 


2.8 


Survey impact: survey sample 


94.7 


91.9 


2.9 * 


3.1 


Grand Rapids Human Capital Development 


Records impact: survey sample 


90.3 


88.4 


1.9 


2.1 


Survey impact: survey sample 


93.6 


91.9 


1.7 


1.9 


Riverside Labor Force Attachment 


Records impact: survey sample 


74.3 


65.0 


9 3 *** 


14.3 


Survey impact: survey sample 


85.7 


80.8 


4 9 ♦♦ 


6.0 


Riverside Human Capital Development 


Records impact: survey sample 


69.3 


60.0 


9.2 *** 


15.4 


Survey impact: survey sample 


82.8 


74.2 


8.6 *** 


11.6 


Portland 


Records impact: survey sample 


84.6 


84.0 


0.5 


0.6 


Survey impact: survey sample 


93.5 


93.3 


0.2 


0.2 



SOURCES: MDRC calculations from Five-Year Client Survey and state and county administrative records. 
NOTES: See Appendix A. 
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3 and 4 percentage points higher than were recorded with UI earnings data, but impacts for LFAs 
and HCDs were similar when calculated with either data source. The next best result occurs 
when both program and control group members have similar rates of discrepant reporting, be- 
cause impacts estimated from UI earnings and survey data will be similar. This situation is dem- 
onstrated by results for Riverside HCD and Portland. As shown in Table H.2, the survey records 
higher employment levels than do UI earnings (especially in Riverside); but differences are con- 
sistent for program and control groups, leaving impact levels nearly unchanged. 

Variation in rates of discrepant reporting by research group is more problematic, because 
it affects impact results. As shown in Table H.2, this result occurred only for Riverside UFA. In 
Riverside, the proportion of control group members who ever worked for pay was nearly 16 per- 
centage points higher when recorded from survey responses than UI earnings data. LFAs also re- 
ported a higher incidence of employment on the survey, but the discrepancy was not as great (11 
percentage points). As a result, survey impacts on employment were more than 4 percentage 
points smaller than UI earnings impacts for the survey respondent sample. Nonetheless, both 
sources record a statistically significant impact on employment for Riverside LFA.^ 

The results for year 5 are more problematic, in that most programs show at least a small 
positive difference in employment when calculated with survey data, but not when calculated 
with UI earnings (Table H.3). The reasons for this discrepancy differ by site and program. In 
general, program group members were more likely than control group members to have higher 
levels of employment recorded on the survey and, correspondingly, less likely to have higher lev- 
els of employment recorded on UI earnings. The difference is most extreme in Portland, where 
the program-control group difference changes from a loss in employment when calculated with 
UI records to a gain when calculated with survey data, although neither difference was statisti- 
cally significant.’ 



^The survey results for Riverside show that joblessness was not as pervasive as indicated by UI data. Interest- 
ingly, in all four sites respondents’ most recent jobs that were reported only on the survey provided fewer hours of 
work per week on average and lower hourly pay and were much less likely to provide medical coverage than jobs 
that were reported on both sources. This nonexperimental finding suggests that sample members in Riverside relied 
more on self-employment, household employment, and service jobs for small employers than those in the other sites 
in the evaluation. This result is still consistent with the overall characterization of Riverside as having a weaker labor 
market than the other sites in the evaluation. 

^The p-value of the UI earnings impact was .13. 
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National Evaluation of Welfare-to-Work Strategies 
Appendix Table H.3 

Comparison of Impact Estimates from Survey and UI Earnings 
Data for Employment in Year 5 



Site and Program 


Program 
Group (%) 


Control 
Group (%) 


Difference 

(Intact) 


Percentage 
Change (%) 


Atlanta Labor Force Attachment 
Records impact: survey sample 


68.1 


Ever emoloved in vear 5 
69.5 -1.3 


-1.9 


Survey impact: survey sample 


69.0 


65.2 


. 3.7 


5.7 


Atlanta Human Capital Development 


Records impact: survey sample 


69.2 


69.5 


-0.3 


-0.5 


Survey impact: survey sample 


64.8 


65.2 


-0.4 


-0.6 


Grand Rapids Labor Force Attachment 


Records impact: survey sample 


74.7 


74.6 


0.1 


0.1 


Survey impact: survey sample 


79.5 


75.7 


3.8 


5.0 


Grand Rapids Human Capital Development 


Records impact: survey sample 


72.0 


74.6 


-2.6 


-3.5 


Survey impact: survey sample 


77.7 


75.7 


2.1 


2.7 


Riverside Labor Force Attachment 


Records impact: survey sample 


49.1 


45.7 


3.4 


7.5 


Survey impact: survey sample 


63.8 


57.6 


6.2 ** 


10.8 


Riverside Human Capital Development 


Records impact: survey sample 


47.8 


41.7 


6.1 * 


14.7 


Survey impact: survey sample 


58.0 


49.2 


8.8 ** 


17.8 


Portland 


Records impact: survey sample 


57.6 


64.2 


-6.7 


-10.4 


Survey impact: survey sample 


60.9 


56.0 


4.9 


8.7 



SOURCES: MDRC calculations from Five-Year Client Survey and state and county administrative records. 
NOTES: See Appendix A. 
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Appendix I 



Impacts on Employment, Welfare, Income, 
and Related Outcomes for the Client Survey, Child 
Outcomes Study Survey, and Teacher Survey Samples 



This appendix presents a brief summary of program impacts on employment, earnings, 
welfare payments, combined income, and other outcomes calculated with administrative records 
and survey responses for the three survey samples. It then compares program impacts for each 
sample with impacts calculated for the larger samples from which its members were selected, 
including (for administrative data) the full impact sample. These findings provide context for in- 
terpreting program impacts on other outcomes measured for each survey sample, including child 
care use and cost and child outcomes. 

Table I.l (for administrative records data) and Table 1.2 (for survey data) show the control 
means and impacts for the three survey samples and (for Table I.l) for the full impact sample. 
There are several reasons why impacts for a particular outcome may vary by sample. Samples 
may be chosen from different months of random assignment or selected from different sub- 
groups. In addition, response rates may vary for different samples (or may differ by research 
group within a sample), and the samples may differ in the degree to which certain subgroups may 
be over- or underrepresented among respondents. This appendix does not attempt to explain 
which source of variation accounts for differences in impacts across samples.' 

As discussed below, for Riverside HCDs impacts are larger and more positive in each sur- 
vey sample than for the full impact sample. In addition, effects were generally more positive for the 
COS sample and, especially, the Teacher Survey sample than for the full impact sample. In contrast, 
impact results were less positive for Portland’s Client Survey respondents than for the full impact 
sample. 



I. Impacts for the Five-Year Client Survey Sample 

A. Administrative Records Outcomes 

For the Client Survey sample, most programs did not increase job finding (percentage 
ever employed) over five years, but both programs in Atlanta and Riverside led to impacts on 
earnings of between $2,500 and $3,500. The other programs led to positive differences in total 
earnings that were small and not statistically significant. All programs also reduced months of 
welfare receipt and total payments over five years, though the Atlanta HCD-control group differ- 
ences on these measures were small and not statistically significant. Impacts on combined in- 
come over five years showed greater variation by program. Both Atlanta programs led to small 
increases in income, whereas both Grand Rapids programs reduced income by a small amount. 
(None of these differences were statistically significant.) Relatively large losses in income (be- 
tween $3,000 and $4,000) over five years were found for Riverside LFA and Portland. Riverside 
HCD had almost no effect on income. 



'See Appendix G for a discussion of these issues. 
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SOURCES: MDRC calculations from state and county administrative records. 
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Appendix Table 1.2 

Impacts on Selected Outcomes, Based on Survey Data, by Sample 



Site 


Client 

Survey Sample 


COS Sample 


Teacher 
Survey Sample 


Control 

Group (%) Impact 


Control 

Group (%) Impact 


Control 

Group (%) Impact 




Received education credential since random assignment 




Atlanta Labor Force Attachment 


10.7 


4.8 ♦♦ 


12.8 


0.8 


10.9 


5.4 


Atlanta Human Capital Development 


10.7 


10.9 ♦♦♦ 


12.8 


14.6 *** 


10.9 


j2 9 *** 


Grand Rapids Labor Force Attachment 


26.1 


-7.4 *** 


29.1 ■ 


-11.3 *** 


27.4 - 


11.1 ** 


Grand Rapids Human Capital Development 


26.1 


2.2 


29.1 


2.3 


27.4 


4.2 


Riverside Labor Force Attachment 


19.1 


-0.4 


19.5 


3.2 


19.6 


3.0 


Lacked high school diploma or basic skills 


14.5 


3.0 


14.8 


5.5 


15.9 


7.5 


Riverside Human Capital Development 


14.5 


10.6 ♦♦♦ 


14.8 


14.7 *** 


15.9 


11.9 ** 








Employed at 


interview 






Atlanta Labor Force Attachment 


54.3 


2.9 


59.6 


-0.8 


63.7 


-1.6 


Atlanta Human Capital Development 


54.3 


-1.4 


59.6 


0.3 


63.7 


-3.0 


Grand Rapids Labor Force Attachment 


67.7 


-2.6 


67.2 


3.9 


64.9 


8.8 


Grand Rapids Human Capital Development 


67.7 


-1.5 


67.2 


-1.4 


64.9 


7.2 


Riverside Labor Force Attachment 


48.9 


6.1 ♦♦ 


44.7 


6.2 


50.0 


6.8 


Lacked high school diploma or basic skills 


43.0 


5.9 


37.3 


9.7 ♦ 


44.9 


9.2 


Riverside Human Capital Development 


43.0 


8.0 ♦♦ 


37.3 


13.5 ♦♦ 


44.9 


8.5 






Emnloved full time at interview 




Atlanta Labor Force Attachment 


45.5 


1.7 


51.6 


0.3 


55.0 


-2.2 


Atlanta Human Capital Development 


45.5 


-1.4 


51.6 


-0.3 


55.0 


-3.3 


Grand Rapids Labor Force Attachment 


54.3 


-0.8 


55.2 


0.7 


51.7 


6.8 


Grand Rapids Human Capital Development 


54.3 


-1.3 


55.2 


-3.1 


51.7 


7.9 


Riverside Labor Force Attachment 


35.9 


8.2 *** 


29.6 


11.0 *** 


32.3 


13.8 *** 


Lacked high school diploma or basic skills 


31.8 


8.4 ** 


23.7 


15.5 *** 


28.6 


J9 3 *** 


Riverside Human Capital Development 


31.8 


4.9 


23.7 


9.0 * 


28.6 


6.5 






Employed and using child care at interview 




Atlanta Labor Force Attachment 


21.5 


0.8 


38.9 


1.9 


42.4 


0.5 


Atlanta Human Capital Development 


21.5 


-1.1 


38.9 


-2.0 


42.4 


-3.7 


Grand Rapids Labor Force Attachment 


30.7 


4.3 * 


49.5 


2.4 


47.4 


6.6 


Grand Rapids Human Capital Development 


30.7 


1.4 


49.5 


-0.7 


47.4 


6.7 


Riverside Labor Force Attachment 


26.6 


0.8 


35.1 


3.3 


41.1 


2.5 


Lacked high school diploma or basic skills 


25.2 


0.3 


31.5 


3.3 


40.7 


-1.3 


Riverside Human Capital Development 


25.2 


-0.2 


31.5 


2.1 


40.7 


-8.5 
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Ajppendix Table 1.2 (continued) 



Site 


Client 

Survey Sample 


COS Sample 


Teacher 
Survey Sample 


Control 

Group (%) Impact 


Control 

Group (%) Impact 


Control 

Group (%) Inmact 






Respondent and all children had medical 








coverage in month before interview 




Atlanta Labor Force Attachment 


71.1 


-1.0 


74.3 


-1.5 


77.8 


-7.5 ♦ 


Atlanta Human Capital Development 


71.1 


0.2 


74.3 


0.2 


77.8 


0.0 


Grand Rapids Labor Force Attachment 


70.9 


-1.7 


76.4 


-1.7 


74.7 


-0.3 


Grand Rapids Human Capital Development 


70.9 


-1.2 


76.4 


-4.3 


74.7 


0.9 


Riverside Labor Force Attachment 


77.6 


-1.9 


83.7 


-3.1 


85.5 


-3.0 


Lacked high school diploma or basic skills 


77.6 


0.2 


84.6 


0.4 


85.7 


3.1 


Riverside Human Capital Development 


77.6 


0.8 


84.6 


-2.9 


85.7 


-3.8 






Married and living with spouse 




Atlanta Labor Force Attachment 


8.4 


1.3 


7.7 


2.3 


7.3 


2.7 


Atlanta Human Capital Development 


8.4 


-1.5 


7.7 


0.9 


7.3 


2.0 


Grand Rapids Labor Force Attachment 


20.5 


2.3 


22.8 


2.6 


24.0 


3.4 


Grand Rapids Human Capital Development 


20.5 


-0.2 


22.8 


1.1 


24.0 


-5.0 


Riverside Labor Force Attachment 


22.0 


-1.4 


21.2 


2.4 


25.6 


-0.8 


Lacked high school diploma or basic skills 


18.1 


0.5 


18.4 


2.7 


21.6 


-1.5 


Riverside Human Capital Development 


18.1 


3.7 


18.4 


0.8 


21.6 


0.4* 




Gave birth to another child since random assignment 




Atlanta Labor Force Attachment 


12.4 


-0.8 


20.5 


-1.4 


24.3 


-3.7 


Atlanta Human Capital Development 


12.4 


0.1 


20.5 


0.1 


24.3 


-0.5 


Grand Rapids Labor Force Attachment 


21.7 


0.9 


29.4 


-2.2 


32.6 


-9.2 ♦ 


Grand Rapids Human Capital Development 


21.7 


0.5 


29.4 


0.4 


32.6 


-2.8 


Riverside Labor Force Attachment 


22.1 


3.4 


31.0 


10.4 


32.4 


6.6 


Lacked high school diploma or basic skills 


23.1 


5.1 * 


34.0 


9.6 ** 


37.5 


1.8 


Riverside Human Capital Development 


23.1 


1.0 


34.0 


-0.7 


37.5 


-5.7 




Livim 


i with another wage earner in month before interview 


Atlanta Labor Force Attachment 


27.6 


1.5 


24.2 


3.6 


29.3 


-2.2- 


Atlanta Human Capital Development 


27.6 


2.6 


24.2 


4.2 


29.3 


1.9 


Grand Rapids Labor Force Attachment 


43.0 


3.8 


41.4 


4.7 


42.8 


5.8 


Grand Rapids Human Capital Development 


43.0 


-2.9 


41.4 


2.3 


42.8 


-1.3 


Riverside Labor Force Attachment 


42.2 


-4.7 ♦ 


38.2 


-2.6 


39.6 


-4.7 


Lacked high school diploma or basic skills 


39.7 


-7.1 ♦♦ 


37.2 


-6.0 


37.1 


-5.3 


Riverside Human Capital Development 


39.7 


-0.1 


37.2 


-0.9 


37.1 


-2.6 
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Appendix Table 1.2 (continued) 



Client Teacher 

Survey Sample COS Sample Survey Sample 

Control Control Control 

Site Group (%) Impact Group (%) Impact Group (%) Impact 

At least one household member received $1,000 
or more in earnings 



Atlanta Labor Force Attachment 


9.8 


0.4 


11.8 


0.9 


15.2 


-3.0 


Atlanta Human Capital Development 


9.8 


-0.8 


11.8 


3.5 


15.2 


2.7 


Grand Rapids Labor Force Attachment 


21.9 


0.7 


26.4 


1.7 


27.4 


0.4 


Grand Rapids Human Capital Development 


21.9 


-0.7 


26.4 


-0.8 


27.4 


-1.6 


Riverside Labor Force Attachment 


21.1 


-0.7 


20.5 


1.4 


22.6 


-2.1 


Lacked high school diploma or basic skills 


17.5 


-2.5 


17.3 


-2.8 


17.2 


-1.6 


Riverside Human Capital Development 


17.5 


2.6 


17.3 


0.9 


17.2 


0.1 



SOURCE: MDRC calculations from the Five-Year Client Survey, Child Outcomes Study, and Teacher Survey. 



NOTES: See Appendix A.2. 



The pattern of impacts for the Client Survey sample resembled impacts for the full impact 
sample for both programs in Atlanta and Grand Rapids, but varied substantially for Riverside 
LFA and HCD and for Portland. For Riverside LFA, employment and earnings impacts were 
similar for the two samples, but welfare reductions were considerably larger for the Client Survey 
sample. As a result. Riverside LFAs in the survey sample show a much larger reduction in com- 
bined income over five years than those in the full impact sample. A large disparity in impacts 
was found for Riverside HCD as well, but in the opposite direction. For the Client Survey sam- 
ple, the program led to substantial gains in total earnings over five years ($3,503) and in stable 
employment in year 5 (7.1 percentage points); these effects were not seen for the full impact 
sample. Reductions in welfare were similar for the two samples, however. Therefore, the effect 
on combined income is more positive for Riverside HCDs in the Client Survey sample. Finally, 
the Portland program had little effect on total earnings over five years for the Client Survey sam- 
ple, but unusually large increases for the full impact sample. Moreover, the Portland program led 
to a relatively large decrease in combined income for the Client Survey sample, but a small (and 
not statistically significant) increase for the full impact sample. Differences for these three pro- 
grams should be kept in mind when interpreting impacts on other outcomes. 

B. Survey Outcomes 

Table 1.2 shows program impacts on a selection of outcomes for parents and families that 
may indirectly affect the well-being of children. As discussed in the text, programs led to rela- 
tively few impacts on these measures for the Client Survey sample. Of note, the Atlanta and Riv- 
erside HCD programs led to relatively large increases in attainment of an education credential 
after random assignment and both Riverside programs increased employment at the end of five 
years. Grand Rapids LFA did not increase employment at the end of year 5, but led to a small 
increase in the percentage of Client Survey respondents who used child care for employment. 



II. Impacts for the Five-Year Child Outcomes Study sample 

A. Administrative Records Outcomes 

Most programs led to positive impacts on employment or on earnings for COS sample — 
although some differences were not statistically significant. Both Riverside programs raised av- 
erage earnings by more than $6,000 per program group member, an unusually large amount. 
These two programs and the Grand Rapids LFA program also led to impacts on stable employ- 
ment in year 5. All programs reduced months on welfare and total payments over five years, but 
effects were small and not statistically significant for both Atlanta programs and for Grand Rap- 
ids HCD. Impacts on combined income were mixed. The Riverside HCD program and both At- 
lanta programs led to small and not statistically significant increases in income, whereas both 
Grand Rapids programs and the Riverside LFA program had almost no effect. 

To some extent, the pattern of impact is more positive for the COS sample than for the 
Client Survey and full impact samples, particularly for the measure of combined income. The 
biggest differences in impact estimates are for the two Riverside programs, especially Riverside 
HCD. That program’s five-year earnings gains for the COS sample exceeded impacts for the full 
impact sample by nearly $5,000 per sample member. The difference in impacts on combined in- 
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come was nearly as large. For Grand Rapids LFAs, results were also more positive for the COS 
sample than for the Client Survey or full impact sample.^ 

B. Survey Outcomes 

For the COS sample, the pattern of impacts on survey outcomes looks similar to impacts 
for all respondents to the Client Survey. The two Riverside programs led to the most positive re- 
sults. They increased employment and employment with full-time hours at the end of five years. 
Riverside HCD also increased degree attainment by nearly 1 5 percentage points above the con- 
trol group level. 

Less positively, Riverside LFA increased the proportion of COS sample members who 
gave birth to another child after random assignment by 10 percentage points above the control 
group average. Only a small difference was found when all Client Survey respondents in River- 
side were included in the calculation. 



III. Impacts for the Teacher Survey Sample 

A. Administrative Records Outcomes 

In general, COS families who participated in the Teacher Survey experienced the most 
positive impacts of the four research samples. All programs increased five-year earnings above 
control group levels, although some differences were small and not statistically significant. 
Moreover, impacts on earnings averaged more than $800 per year for the three LFA programs 
and for Riverside HCD. Most programs also increased stable employment in year 5. All pro- 
grams reduced total welfare payments over five years, but effects were small and not statistically 
significant for Atlanta and Grand Rapids HCD. Over five years, all programs except Riverside 
LFA increased combined income, although effects were small and not statistically significant in 
most programs. 

The disparity of impact results across samples was greatest for Atlanta and Grand Rapids 
LFA. These programs led to much more positive effects on earnings, on stable employment, and 
on combined income for the Teacher Survey sample, than for the other samples. Furthermore, as 
for the COS sample. Riverside HCD led to much larger impacts on stable employment and earn- 
ings and more positive effects on combined income for the Teacher Survey sample than for the 
full impact and Client Survey samples. 

B. Survey Outcomes 

For survey outcomes, more program-control group differences of at least moderate size 
(+/- 5 percentage points) were found for the Teacher Survey sample than for the other survey 
samples. However, most of these differences were not statistically significant. As for the other 
samples, both Riverside programs increased employment at the end of year 5 above control group 
levels. In addition, a higher percentage of LFAs gave birth to a child during the follow-up, an ef- 

^It should be remembered that both the full iirpact and Client Survey samples include parents whose youngest 
child was aged 6 or over at random assignment, but the COS sample does not. 
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feet also seen for all LFAs in the Riverside COS sample. Unlike its effect on the COS and Client 
Survey samples, Riverside HCD decreased below control group levels the proportion of Teacher 
Survey sample members who were working and using child care at the end of year 5 and de- 
creased the incidence of childbearing after random assignment.^ 

The Grand Rapids LFA and HCD programs also led to larger effects for the Teacher Sur- 
vey sample than for the COS samples, although not all differences were statistically significant. 
Each program increased employment and employment with full-time hours at the end of year 5. 
Grand Rapids LFAs were also more likely than control group members to be livirig with another 
person who was working for pay, more likely to be using child care for employment at the end of 
year 5, and less likely to have given birth to a child after random assignment. The Grand Rapids 
HCD program also increased child care use for employment at the end of year 5 and decreased 
the incidence of being married and living with spouse. 

Finally, programs had as little effect on medical coverage at the end of year 5 for the 
Teacher Survey sample as they did for the other survey samples. However, Atlanta LFA led to a 
statistically significant decrease in coverage, a result not seen for the other samples. 



^For Riverside LFA, only the impact on full-time employment was statistically significant. 
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National Evaluation of Welfare-to-Work Strategies 
Appendix Table J. 1 
Child Outcome Measures 



Measure 


Source 


For Whom 
Available 


Description 


Social Skills and Behavior^ 








Externalizing behavior 


Mother 


Focal child 


Includes such behaviors as fighting and 
arguing with others. Scores could range 
from 0 to 18. Cronbach’s alpha of .78. 




Teacher 


Focal child 


Includes behaviors such as fighting and 
arguing. Scores could range from 0 to 
15. Cronbach’s alpha of .89. 


Internalizing behavior 


Mother 


Focal child 


Includes such behaviors as acting sad 
and being anxious about being with 
others. Scores could range from 0 to 24. 
Cronbach’s alpha of .66. 




Teacher 


Focal child 


Includes such behaviors as acting sad 
and being anxious about being with 
others. Scores could range from 0 to 18. 
Cronbach’s alpha of .75. 


Hyperactivity 


Mother 


Focal child 


Includes such behaviors as disrupting 
others and acting without thinking. 
Scores could range from 0 to 18. 
Cronbach’s alpha of .76. 




Teacher 


Focal child 


Includes such behaviors as disturbing 
others and acting without thinking. 
Scores could range from 0 to 6. 
Cronbach’s alpha of .73. 


Cooperation 


Child 


Focal child 


Includes such behaviors as listening to 
adults and finishing schoolwork and 
homework on time. Scores could range 
from 0 to 30. Cronbach’s alpha of .77. 




Mother 


Focal child 


Includes items addressing neatness, 
helpfulness at home, and giving of 
compliments. Scores could range from 0 
to 39. Cronbach’s alpha of .85. 
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Measure 


Source 


For Whom 
Available 


Description 




Teacher 


Focal child 


Includes such school-related behaviors 
as staying focused and completing work. 
Scores could range from 0 to 27. 
Cronbach’s alpha of .92. 


Positive assertion 


Child 


Focal child 


Includes ratings by children themselves 
on the frequency with which they 
demonstrate behaviors such as being 
active with other children and properly 
handling children who are behaving 
inappropriately. Scores could range 
from 0 to 30. Cronbach’s alpha of .71. 




Mother 


Focal child 


Includes frequency with which children 
demonstrate such skills as initiating 
interaction with other children and 
making friends. Scores could range from 
0 to 30. Cronbach’s alpha of .83. 




Teacher 


Focal child 


Includes frequency with which children 
show such skills as interacting well with 
other children and making friends. 
Scores could range from 0 to 18. 
Cronbach’s alpha of .84. 


Self-control 


Child 


Focal child 


Includes ratings by children themselves 
on the frequency with which they 
demonstrate behaviors such as 
controlling anger and properly handling 
children who are behaving 
inappropriately. Scores could range 
from 0 to 30. Cronbach’s alpha of .69. 




Mother 


Focal child 


Includes such behaviors as speaking 
politely and responding appropriately to 
conflict. Scores could range from 0 to 
30. Cronbach’s alpha of .78. 




Teacher 


Focal child 


Includes such behaviors as responding 
appropriately to conflict and interacting 
well with peers. Scores range from 0 to 
27. Cronbach’s alpha of .92. 


Empathy 


Child 


Focal child 


Includes such behaviors as listening to 
others and trying to understand others’ 
feelings. Scores could range from 0 to 
30. Cronbach’s alpha of .79. 
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Measure 


Source 


For Whom 
Available 


Description 


Responsibility 


Mother 


Focal child 


Includes such behaviors as speaking 
politely to others and listening to others. 
Scores could range from 0 to 27. 
Cronbach’s alpha of .80. 


Interpersonal skills 


Teacher 


Focal child 


Includes such skills being sensitive to 
others and making friendships. Scores 
could range from 0 to 12. Cronbach’s 
alpha of .77. 


Positive approaches to 
learning 


Teacher 


Focal child 


Includes such habits as working 
carefully and being creative. Scores 
could range from 0 to 18. Cronbach’s 
alpha of .85. 


Academic Functionine 


School engagement 


Child'’ 


Focal child 


From 0 to 3 on seven items measuring 



school engagement, where 0 = “never 
true”; 1 = “sometimes true”; 2 = “often 



true”; 3 = “very often true.” Scores 
therefore range from 0 to 21 , with 
appropriate items reverse-coded so that 
higher scores indicate more engagement 
in schools. The items are: 

“When I’m in school, I feel happy.” 

“I work very hard in school.” 

“When I’m in school, I feel bored.” 
“When I’m in school, I pay attention 
in class.” 

“I try to learn as much as I can about 
my school subjects.” 

“When I’m in class, I just pretend to 
work.” 

“When I’m in class, I try very hard.” 
Cronbach’s alpha of .75. 

Teacher Focal child Where 1 = “very true”; 2 = “sort of 

true”; 3 = “not very true”; 4 = “not at all 
true”: 

“This student . . . 

Comes to class unprepared.” 

Pays attention.” 

Works as hard as s/he can.” 

Does more than is required.” 

Is interested in schoolwork outside of 
the classroom.” 



(continued) 




-442- 



77 



Measure 


Source 


For Whom 
Available 


Description 








Thinks about other things in class.” 
“In my class, this student seems . . .A 
million miles away.” 

Interested.” 

Tuned in.” 

Sleepy.” 

Angry.” 

Responses to the items were recoded 
and given values from 0 to 3. Scores 
therefore range from 0 to 33. 
Cronbach’s alpha of .92. 


Suspended or expelled since 
last interview (%) 


Mother 


Each Child 


“Since the last interview, (has this 
child/have any of these children) ever 
been suspended, excluded, or expelled 
from school?” 


Had disciplinary action taken 
weekly (%) 


Teacher 


Focal child 


“In the last month or two, how often 
have you taken any disciplinary action 
regarding this student? (1) Several times 
a week; (2) About once a week; (3) One 
to three times a month; (4) Once in 
awhile, less than once a month; (5) 
Never.” Categories (1) and (2) were 
collapsed and the responses were 
dichotomized to determine whether 
teachers reported that disciplinary action 
had taken place weekly. 


Had a discipline problem 
that resulted in parent(s) 
being notified this school 
year (%) 


Teacher 


Focal child 


“So far this school year, has this student 
had any behavior or discipline problem 
at school that resulted in his/her 
parent(s) or guardian(s) being sent a 
note or being asked to come in to meet 
with you or the principal? (1) No, never; 
(2) Yes, once; (3) Yes, more than once.” 
Categories (2) and (3) were collapsed 
and the responses were dichotomized to 
determine whether focal children had a 
discipline problem that resulted in their 
parents being notified this school year. 


Dropped out of school (%) 


Mother 


Each Child 


“(Has this child/Have any of these 
children) ever dropped out of school?” 
Analyzed for children aged 10 to 24 at 
the five-year follow-up. 
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Measure 


Source 


For Whom 
Available 


Description 


Broad Math Score on the 
Woodcock-Johnson Tests of 
Achievement-Revised*^ 


Assessment 


Focal child 


Administered the Calculation and 
Applied Problems tests. Composite 
score is age-standardized, with a mean 
of 100 and a standard deviation of 15. 


Above-average Broad Math 
Score (%) 


Assessment 


Focal child 


Focal children with Broad Math scores 
higher than 1 10 were considered to have 
above average scores. 


Below-average Broad Math 
Score (%) 


Assessment 


Focal child 


Focal children with Broad Math scores 
lower than 90 were considered to have 
below average scores. 


Broad Reading Score on the 
Woodcock-Johnson Tests of 
Achievement-Revised*^ 


Assessment 


Focal child 


Administered the Passage 
Comprehension and Letter-Word 
Identification tests. Composite score is 
age-standardized, with a mean of 100 
and a standard deviation of 15. 


Above-average Broad 
Reading Score (%) 


Assessment 


Focal child 


Focal children with Broad Reading 
scores higher than 110 were considered 
to have above average scores. 


Below-average Broad 
Reading Score (%) 


Assessment 


Focal child 


Focal children with Broad Reading 
scores lower than 90 were considered to 
have below average scores. 


Repeated a grade (%) 


Mother 


Each Child 


“Since the last interview, (has this 
child/have any of these children) 
repeated a grade for any reason?” 




Teacher 


Focal child 


“Not counting kindergarten, has this 
student repeated any grades or been 
retained in grade for any reason?” 


Below grade level, math (%) 


Teacher 


Focal child 


“Is this student currently performing 
above grade level, below grade level, or 
right around grade level in reading and 
math?” Responses were coded so as to 
obtain the measure of whether the 
student was below grade level in math. 


Above grade level, math (%) 


Teacher 


Focal child 


“Is this student currently performing 
above grade level, below grade level, or 
right around grade level in reading and 
math?” Responses were coded so as to 
obtain the measure of whether the 
student was above grade level in math. 
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Measure 


Source 


For Whom 
Available 


Description 


Below grade level, 
reading (%) 


Teacher 


Focal child 


“Is this student currently performing 
above grade level, below grade level, or 
right around grade level in reading and 
math?” Responses were coded so as to 
obtain the measure of whether the 
student was below grade level in 
reading. 


Above grade level, 
reading (%) 


Teacher 


Focal child 


“Is this student currently performing 
above grade level, below grade level, or 
right around grade level in reading and 
math?” Responses were coded to obtain 
the measure of whether the student was 
above grade level in reading. 


In remedial math group (%) 


Teacher 


Focal child 


“What level-of-ability group is this 
student in for mathematics? (1) High- 
ability group, (2) Middle-ability group, 
(3) Low-ability (or remedial) group, (4) 
No grouping by ability, or 
heterogeneous grouping.” Responses 
were dichotomized to obtain the 
measure of whether the student was in a 
remedial group for math. 


In rerriedial reading 
group (%) 


Teacher 


Focal child 


“What level-of-ability group is this 
student in for reading? (1) High-ability 
group, (2) Middle-ability group, (3) 
Low-ability (or remedial) group, (4) No 
grouping by ability, or heterogeneous 
grouping.” Responses were 
dichotomized to obtain the measure of 
whether the student was in a remedial 
group for reading. 


Attended a special class for 
physical, emotional, or 
mental condition (%) 


Mother 


Each child 


“(Does your child/Do any of your 
children) go to a special class or special 
school, or get special help in school for 
any physical, emotional, or mental 
condition?” 
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Measure 


Source 


For Whom 
Available 


Description 


Children identified as 
needing and receiving 
special services (%) 


Teacher 


Focal child 


Services included Title I services, 
English as a Second Language (ESL) or 
bilingual instruction, remedial 
instruction, speech therapy, or a 
resource for the gifted. Responses were 
dichotomized such that focal children 
were classified as needing and receiving 
special services if teachers reported they 
needed and received any of the services. 


Children identified as 
needing and not receiving 
special services (%) 


Teacher 


Focal child 


Services included Title I services, 
English as a Second Language (ESL) or 
bilingual instruction, remedial 
instruction, speech therapy, or a 
resource for the gifted. Responses were 
dichotomized such that focal children 
were classified as needing and not 
receiving special services if teachers 
reported they needed but did not receive 
any of the services listed. The measures 
of needing and receiving and needing 
and not receiving are not mutually 
exclusive; children could have been 
identified as needing and receiving some 
services while needing and not receiving 
other services. 


Days child had been absent 
during current school 
year (%) 


Teacher 


Focal child 


“So far this school year, about what 
percentage of the days that this student 
has been enrolled in your classroom has 
he/she been absent for any reason 
including sickness?” 


Days child had been tardy 
during current school 
year (%) 


Teacher 


Focal child 


“So far this school year, about what 
percentage of the days that this student 
has been enrolled in your classroom has 
he/she been tardy for any reason 
including sickness?” 


Health and Safetv 


General health rating 


Mother 


Focal child 


“Would you say that your child’s health 
in general is: (1) Excellent, (2) Very 
good, (3) Good, (4) Fair, (5) Poor?” 


In very good or excellent 
health (%) 


Mother 


Focal child 


Responses to the general health rating 
item were dichotomized so as to obtain 
the measure of whether the focal child 
was in very good or excellent health. 
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Measure 


Source 


For Whom 
Available 


Description 


Had a physical, emotional, or 
mental condition that 
required frequent medical 
attention (%) 


Mother 


Each child 


“(Does your child/Do any of your 
children) have a physical, emotional, or 
mental condition that requires frequent 
medical attention, frequent use of 
medication, or the use of any special 
equipment such as a wheelchair or a 
breathing mask?” It was specified that 
this does not include eyeglasses. 


Had a physical, emotional, or 
mental condition that 
impeded on mother’s ability 
to go to work/school (%) 


Mother 


Each child 


“(Does your child/Do any of your 
children) have a physical, emotional, or 
mental condition that demands a lot of 
your attention and makes it hard for you 
to go to school or work?” 


Accident, injury, or 
poisoning that required an 
emergency room visit (%) 


Mother 


Each child 


“Since the last interview, (has your 
child/have any of your children) had an 
' accident, injury, or poisoning requiring a 
visit to a hospital emergency room or 
clinic?” 


Other Outcomes 


Did not live with parent, 
since last interview, because 
parent could not care for 
child (%) 


Mother 


Each child 


“Since the last interview, (has your 
child/have any of your children) not 
lived with you because you couldn’t 
care for (him/her/them)?” 


Ever had a baby as 
a teen (%) 


Mother 


Each child 


“(Has your child/Have any of your 
children) had a baby?” Analyzed for 
children aged 14 to 24 at the five-year 
follow-up, though analysis was limited 
to those having had a baby as a teenager. 



NOTES: Some items were asked of each child in the family while others were asked of only the focal child. The “for 
whom available” column notes for which group of children each item was asked. The items asked of each child are 
reported in Chapters 1 1 and 12. The items asked of the focal child only are reported only in Chapter 12. Cronbach’s alphas 
are reported on the mother/child Child Outcomes Study sample (N = 2,332) for the mother- and child-report items and on 
the teacher sample (N = 1,472) for the teacher-report items. 

“Items adapted from Gresham and Elliot, 1990. In the social skills domain, focal children responded to items with 0 
= “Never”; 1 = “Sometimes”; 2 = “Often”; or 3 = “Very often.” Mothers also responded to items v^th 0 = “Never” to 3 = 
“Very often.”. Teachers responded to items with 1 = “Never”; 2 = “Sometimes”; 3 = “Often”; or 4 = “Very often”; 
responses were recoded from 0 to 3. 

^The child school engagement scale is by Connell, 1990. 

^Woodcock and Johnson, 1989, 1990. 
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